Image recognition method and apparatus utilizing edge detection based on magnitudes of color vectors expressing color attributes of respective pixels of color image

ABSTRACT

An image recognition apparatus operates on data of a color image to obtain an edge image expressing the shapes of objects appearing in the color image, the apparatus including a section for expressing the color attributes of each pixel of the image as a color vector, in the form of a set of coordinates of an orthogonal color space, a section for applying predetermined arrays of numeric values as edge templates to derive for each pixel a number of edge vectors each corresponding to a specific edge direction, with each edge vector obtained as the difference between weighted vector sums of respective sets of color vectors of two sets of pixels which are disposed symmetrically opposing with respect to the corresponding edge direction, and a section for obtaining the maximum modulus of these edge vectors as a value of edge strength for the pixel which is being processed. By comparing the edge strength of a pixel with those of immediately adjacent pixels and with a predetermined threshold value, a decision can be reliably made for each pixel as to whether it is actually located on an edge and, if so, the direction of that edge.

BACKGROUND OF THE INVENTION

[0001] 1. Field of Application

[0002] The present invention relates to an image recognition method andan image recognition apparatus for use in an image recognition system,for extracting from a color image the shapes of objects which are to berecognized. In particular, the invention relates to an image recognitionapparatus which provides a substantial improvement in edge detectionperformance when applied to images such as aerial photographs orsatellite images which exhibit a relatively low degree of variation inintensity values.

[0003] 2. Description of Prior Art

[0004] In the prior art, various types of image recognition apparatusare known, which are intended for various different fields ofapplication. Typically, the image recognition apparatus may be requiredto extract from an image, such as a photograph, all objects having ashape which falls within some predetermined category.

[0005] One approach to the problem of increasing the accuracy of imagerecognition of the contents of photographs is to set the camera whichtakes the photographs in a fixed position and to fix the lightingconditions etc., so that the photographic conditions are alwaysidentical. Another approach is to attach markers, etc., to the objectswhich are to be recognized.

[0006] However in the case of recognizing shapes within satellite imagesor aerial photographs, such prior art methods of improving accuracycannot be applied. That is to say. the photographic conditions such asthe camera position, camera orientation, weather conditions, etc., willvary each time that a photograph is taken. Furthermore, a single imagemay contain many categories of image data, such as image datacorresponding to building, rivers, streets, etc., so that the imagecontents are complex. As a result, the application of image recognitionto satellite images or aerial photographs is extremely difficult.

[0007] To extract the shapes of objects which are to be recognized, fromthe contents of an image, image processing to detect edges etc., can beimplemented by using the differences between color values (typically,the intensity, i.e., gray-scale values) of the pixels which constitute aregion representing an object which is to be recognized and the colorvalues of the pixels which constitute adjacent regions to these objects.Edge detection processing consists of detecting positions at which thereare abrupt changes in the pixel values, and recognizing such positionsas corresponding to the outlines of physical objects. Various types ofedge detection processing are known. With a typical method, smoothingprocessing is applied overall to the pixel values, then each of thepixels for which the first derivative of the intensity variationgradient within the image reaches a local maximum and exceeds apredetermined threshold value are determined, with each such pixel beingassumed to be located on an edge of an object in the image.Alternatively, a “zero-crossing” method can be applied, e.g., wherebythe zero crossings of the second derivative of the gradient are bedetected to obtain the locations of the edge pixels. With a templatetechnique, predetermined shape templates are compared with the imagecontents to find the approximate positions of objects that are to berecognized, then edge detection processing may be applied to the resultsobtained.

[0008] Although prior art image recognition techniques are generallybased upon intensity values of the pixels of an image, various methodsare possible for expressing the pixel values of color image data. If theHSI (hue, saturation, intensity) color space is used, then any pixel canbe specified in terms of the magnitude of its hue, saturation orintensity component. The RGB (red, green, blue) method is widely usedfor expressing image data, however transform processing can be appliedto convert such data to HSI form, and edge detection processing can thenbe applied by operating on the intensity values which are therebyobtained. HSI information has the advantage of being readilycomprehended by a human operator. In particular, an image can easily bejudged by a human operator as having a relatively high or relatively lowdegree of variation in intensity (i.e., high contrast or low contrast).

[0009] Due to the difficulties which are experienced in the practicalapplication of image recognition processing to satellite images oraerial photographs, it would be desirable to effectively utilize all ofthe color information that is available within such a photograph, thatis to say, to use not only the intensity values of the image but alsothe hue and saturation information contained in the image. However ingeneral with prior art types of edge detection processing, only parts ofthe color information, such as the intensity values alone, are utilized.

[0010] A method of edge detection processing is described in Japanesepatent HEI 6-83962, which uses a zero-crossing method and, employing aHSI color space (referred to therein using the designations L,*C*ab,H*abfor the intensity, saturation and hue values respectively) attempts toutilize not only the intensity values but also hue and saturationinformation. In FIG. 47, diagrams 200, 201, 202, and 203 show respectiveexamples of the results of image recognition, applied to a color pictureof an individual, which are obtained by using that method. Diagram 200shows the result of edge detection processing that is applied using onlythe intensity values of each of the pixels of the original picture,diagram 201 shows the result of edge detection processing that isapplied using only the hue values, and diagram 202 shows the resultobtained by using only the saturation values. Diagram 203 shows theresult that is obtained by combining the results shown in diagrams 200,201 and 203. As can be seen, a substantial amount of noise arises in theimage expressed by the saturation values, and this noise is insertedinto the combined image shown in diagram 203.

[0011] In some cases, image smoothing processing is applied in order toreduce the amount of noise within an image, before performing edgedetection processing, i.e., the image is pre-processed by using asmoothing filter to blur the image, and edge detection processingapplied to the resultant image.

[0012] In order to obtain satisfactory results from edge detectionprocessing which is to be applied to an image such as satellite imagesor aerial photograph, for example to accurately and reliably extract theshapes of specific objects such as roads, buildings etc., from the imagecontents, it is necessary not only to determine the degree of “strength”of each edge, but also the direction along which an edge is oriented. Inthe following, and in the description of embodiments of the inventionand in the appended claims, the term “edge” is used in the sense of aline segment which is used as a straight-line approximation to a part ofa boundary between two adjacent regions of a color image. The term“strength” of an edge is used herein to signify a degree of of colordifference between pixels located adjacent to one side of that edge andpixels located adjacent to the opposite side, while the term “edgedirection” is used in referring to the angle of orientation of an edgewithin the image, which is one of a predetermined limited number ofangles. If the direction of an edge could be accurately determined(i.e., based upon only a part of the pixels which constitute that edge),then this would greatly simplify the process of determining all of thepixels which are located along that edge. That is to say, if the edgedirection could be reliably determined estimated by using only a part ofthe pixels located on that edge, then it would be possible to compensatefor any discontinuities within the edge which is obtained as a result ofthe edge detection processing, so that an output image could begenerated in which all edges are accurately shown as continuous lines.

[0013] However with the method described in Japanese patent HEI 6-83962,only the zero-crossing method is used, so that it is not possible todetermine edge directions, since only each local maximum of variation ofa gradient of a color attribute is detected, irrespective of thedirection along which that variation is oriented. With other types ofedge detection processing such as the object template method, processingof intensity values, hue values and saturation values can be performedrespectively separately, to obtain respective edge directions. Howevereven if the results thus obtained are combined, accurate edge directionscannot be detected. Specifically, the edge directions which result fromusing intensity values, hue values and saturation values may be entirelydifferent from one another, so that accurate edge detection cannot beachieved by taking the average of these results.

[0014] Moreover, in the case of a color image such as a satellite imageor aerial photograph which presents special difficulties with respect toimage recognition, it would be desirable to be able to flexibly adjustthe image recognition processing in accordance with the overall colorcharacteristics of the image that is to be processed. That is to say, itshould be possible for example for a human operator to examine such animage prior to executing image recognition processing, to estimatewhether different objects in the image mainly differ mainly with respectto differences in hue, or whether the objects are mainly distinguishedby differences in gray-scale level, i.e., intensity values. The operatorshould then be able to adjust the image recognition apparatus to operatein a manner that is best suited to these image characteristics, i.e., toextract the edges of objects based on the entire color information ofthe image, but for example placing emphasis upon the intensity values ofpixels, or upon the chrominance values of the pixels, whichever isappropriate. However such a type of image recognition apparatus has notbeen available in the prior art.

[0015] Furthermore, in order to apply image recognition processing to animage whose color data are expressed with respect to an RGB color space,it is common practice to first convert the color image data to a an HSI(hue, saturation, intensity) color space, i.e., expressing the data ofeach pixel as a position within such a color space. This enables a humanoperator to more readily judge the color attributes of the overall imageprior to executing the image recognition processing, and enables suchprocessing to be applied to only the a specific color attribute of eachof the pixels, such as the intensity or the saturation attribute.However if processing is applied to RGB data which contain some degreeof scattering of the color values, and a transform from RGB to HSI colorspace is executed, then the resultant values of saturation will beunstable (i.e., will tend to vary randomly with respect to the correctvalues) within those regions of the image in which the intensity valuesare high, and also within those regions of the image in which theintensity values are low. For example, assuming that each of the red,green and blue values of each pixel is expressed by 8 bits, so that therange of values is from 0 to 255, then in the case of a region of theimage in which the intensity values are low, if any of the red, green orblue values of a pixel within that region should increase by 1, thiswill result in a large change in the corresponding value of saturationthat is obtained by the transform processing operation. Instability ofthe saturation values will be expressed as noise, i.e., spurious edgeportions, in the results of edge detection processing which utilizesthese values. For that reason it has been difficult in the prior art toutilize the color saturation information contained in a color image, inimage recognition processing.

[0016] Furthermore if a substantial degree of smoothing processing isapplied to an image which is to be subjected to image recognition, inorder to suppress the occurrence of such noise, then this has the effectof blurring the image, causing rounding of the shapes of edges and alsomerging together any edges which are located closely mutually adjacent.As a result, the accuracy of extracting edge information will bereduced. Conversely, if only a moderate degree of smoothing processingis applied to the image that is to be subjected to image recognition, orif smoothing processing is not applied to the image, then the accuracyof extraction of shapes from the image will be high, but there will be ahigh level of noise in the results so that reliable extraction of theshapes of the required objects will be difficult to achieve.

[0017] Moreover in the prior art, there has been no simple and effectivemethod of performing image recognition processing to extract the shapesof objects which are to be recognized, which will eliminate varioussmall objects in the image that are not intended to be recognized (andtherefore can be considered to constitute noise) without distorting theshapes of the objects which are to be recognized.

SUMMARY OF THE INVENTION

[0018] It is an objective of the present invention to overcome thedisadvantages of the prior art set out above, by providing an imagerecognition method and image recognition apparatus whereby edgedetection for extracting the outlines of objects appearing in a colorimage can be performed by utilizing all of the color information of thepixels of the color image, to thereby achieve a substantially higherdegree of reliability of detecting those pixels which constitute edgesof objects that are to be recognized than has been possible in the priorart, and furthermore to provide an image recognition method andapparatus whereby, when such an edge pixel is detected, the direction ofthe corresponding edge can also be detected.

[0019] It is a further objective of the invention to provide an imagerecognition method and image recognition apparatus whereby processing toextract the shapes of objects which are to be recognized can beperformed such as to eliminate the respective shapes of small objectsthat are not intended to be recognized, without distorting the shapes ofthe objects which are to be recognized.

[0020] To achieve the above objectives, the invention provides an imagerecognition method and apparatus whereby, as opposed to prior artmethods which are based only upon intensity values, i.e., the gray-scalevalues of the pixels of a color image that is to be subjected to imagerecognition processing, substantially all of the color information(intensity, hue and saturation information) contained in the color imagecan be utilized for detecting the edges of objects which are to berecognized. This is basically achieved by successively selecting eachpixel to be processed, i.e., as the object pixel, and determining, foreach of a plurality of possible edge directions, a vector referred to asan edge vector whose modulus indicates an amount of color differencebetween two sets of pixels which are located on opposing sides of theobject pixel with respect to that edge direction. The moduli of theresultant set of edge vectors are then compared, and the edge vectorhaving the largest modulus is then assumed to correspond to the mostlikely edge on which the object pixel may be located. That largest valueor edge vector modulus is referred to as the “edge strength” of theobject pixel, and the direction corresponding to that edge vector isassumed to be the most likely direction of an edge on which the objectpixel may be located, i.e., a presumptive edge for that pixel.Subsequently, it is judged that the object pixel is actually located onits presumptive edge if it satisifes the conditions that:

[0021] (a) its edge strength exceeds a predetermined minimum thresholdvalue, and

[0022] (b) its edge strength is greater than the respective edgestrength values of the two pixels which are located immediately adjacentto it, on opposing sides with respect to the direction of thatpresumptive edge.

[0023] The above processing can be achieved in a simple manner bypredetermining only a limited number of possible edge directions whichcan be recognized, e.g., 0 degrees (horizontal), 90 degrees (vertical),45 degrees diagonal and −45 degrees diagonal. With the preferredembodiments of the invention, a set of arrays of numeric values referredto as edge templates are utilized, with each edge template correspondingto a specific one of the predetermined edge directions, and with thevalues thereof predetermined such that when the color vectors of anarray of pixels centered on the object pixel are subjected to arraymultiplication by an edge template, the edge vector corresponding to thedirection of that edge template will be obtained as the vector sum ofthe result. The respective moduli of the edge vectors thereby derivedfor each of the possible edge directions are then compared, to find thelargest of these moduli, as the edge strength of the object pixel.

[0024] In that way, since all of the color information contained in theimage can be utilized to perform edge detection, the detection can bemore accurately and reliably performed than has been possible in theprior art.

[0025] According to another aspect of the invention, data expressing thecolor attributes of pixels of a color image which is to be subjected toedge detection processing are first subjected to transform processing toexpress the color attributes of the pixels of the image as respectivesets of coordinates of an appropriate color space, in particular, acolor space in which intensity and chrominance information are expressedby separate coordinates. This enables the color attribute information tobe modified prior to performing edge detection, such as to optimize theresults that will be obtained in accordance with the characteristics ofthe particular color image that is being processed. That is to say, therelative amount of contribution of the intensity values to themagnitudes of the aforementioned color vectors can be increased, forexample. If the color attributes are first transformed into a HSI (hue,saturation, intensity) color space, then since such HSI values aregenerally expressed in polar coordinates, a simple conversion operationis applied to each set of h, s, i values of each pixel to express thecolor attributes as a color vector of an orthogonal color space in whichsaturation information and chrominance information are expressed alongrespectively different coordinate axes, i.e. to express the pixel colorattributes as a plurality of linear coordinates of that color space, andthe edge detection processing is then executed.

[0026] It is known that when image data are transformed from a form suchas RGB color values into an HSI color space, instability (i.e., randomlarge-scale variations) may occur in the saturation values which areobtained as a result of the transform. This instability of saturationvalues is most prevalent in those regions of a color image where theintensity values are exceptionally low, and also in those regions wherethe intensity values are exceptionally high. This is a characteristicfeature of such a transform operation, and causes noise to appear in theresults of edge detection that is applied to such HSI-transformed imagedata and utilizes the saturation information, due to the detection ofspurious edge portions as a result of abrupt changes in saturationvalues between adjacent pixels. However with the present invention, suchinstability of the saturation values can be reduced, by modifying thesaturation values obtained for respective pixels in accordance with themagnitudes of the intensity values which are derived for these pixels.The noise which would otherwise be generated by such instability ofsaturation values can thereby be suppressed, enabling more reliablerecognition of objects in the color image to be achieved.

[0027] According to one aspect of the invention, when a transform intocoordinates of the HSI space has been executed, such reduction ofinstability of the saturation values is then achieved by decreasing thesaturation values in direct proportion to amounts of decrease in theintensity values. Alternatively, that effect is achieved by decreasingthe saturation values in direct proportion to decreases in the intensityvalues from a median value of intensity towards a minimum value (i.e.,black) and also decreasing the saturation values in direct proportion toincreases in the intensity values from that median value towards amaximum value (i.e., white).

[0028] According to another aspect of the invention, when a transforminto coordinates of the HSI space has been executed, such reduction ofinstability of the saturation values is then achieved by utilizing apredetermined saturation value modification function (which varies in apredetermined manner in accordance with values of intensity) to modifythe saturation values. In the case of a transform from the RGB colorspace to the HSI color space, that saturation value modificationfunction is preferably derived based on calculating, for each of thesets of r, g, b values expressing respective points in the RGB colorspace, the amount of actual change which occurs in the saturation values of the corresponding HSI set of transformed h, s, i values in responseto a small-scale change in one of that set of r, g, b values. In thatway, a saturation value modification function can be derived which isbased on the actual relationship between transformed intensity valuesand instability of the corresponding saturation values, and can thus beused such as to maintain the saturation values throughout a color imageat a substantially constant level, i.e., by varying the saturationvalues in accordance with the intensity values such as to appropriatelycompensate in those regions of the color space in which instability ofthe saturation values can occur.

[0029] Noise in the edge detection results, caused by detection ofspurious edge portions, can be thereby very effectively suppressed,enabling accurate edge detection to be achieved.

[0030] According to another aspect, the invention provides an imagerecognition method and apparatus for operating on a region image (i.e.,an image formed of a plurality of regions expressing the shapes ofvarious objects, each region formed of a continuously extending set ofpixels in which each pixel is identified by a label as being containedin that region) to process the region image such as to reduce the amountof noise caused by the presence of various small regions, which are notrequired to be recognized. This is achieved by detecting each smallregion having an area that is less than a predetermined threshold value,and combining each such small region with an immediately adjacentregion, with the combining process being executed in accordance withspecific rules which serve to prevent distortion of the shape of objectsthat are to be recognized. These rules preferably stipulate that each ofthe small regions is to be combined with an immediately adjacent otherregion which (out of all of the regions immediately adjacent to thatsmall region) has a maximum length of common boundary line with respectto that small region. In that way, regions are combined withoutconsideration of the pixel values (of an original color image) withinthe regions and considering only the sizes and shapes of the regions,whereby it becomes possible to eliminate small regions which wouldconstitute “image noise”, without reducing the accuracy of extractingthe shapes of objects which are to be recognized.

[0031] The aforementioned rules for combining regions may furtherstipulate that the combining processing is to be executed repetitively,to operate successively on each of the regions which are below theaforementioned area size threshold value, starting from the smallest ofthese regions, then the next-smallest, and so on. It has been found thatthis provided even greater effectiveness in elimination of image noise,without reducing the accuracy of extracting the shapes of objects whichare to be recognized.

[0032] Alternatively, the region combining processing may be executed onthe basis that the aforementioned rules for combining regions furtherstipulate that, for each of the small regions which are below theaforementioned area size threshold value, the total area of the regionsimmediately adjacent to that small region is to be calculated, and theaforementioned combining processing is then to be executed starting withthe small region for which that adjacent area total is the largest, thenthe small region for which the adjacent area total is the next-largest,and so on in succession for all of these small regions.

[0033] A region image, for applying such region combining processing,can for example be generated by first applying edge detection by an edgedetection apparatus according to the present invention to an originalcolor image, to obtain data expressing an edge image in which only theedges of objects appear, then defining each part of that edge imagewhich is enclosed within a continuously extending edge as a separateregion, and attaching a common identifier label to each of the pixelsconstituting that region.

[0034] More specifically, the present invention provides an imagerecognition method for processing image data of a color image which isrepresented as respective sets of color attribute values or an array ofpixels, to successively operate on each of the pixels as an object pixelsuch as to determine whether that pixel is located on an edge within thecolor image, and thereby derive shape data expressing an edge imagewhich shows only the outlines of objects appearing in the color image,with the method comprising steps of:

[0035] if necessary, i.e., if the color attribute values of the pixelsare not originally expressed as sets of coordinates of an orthogonalcolor space such as an RGB (red, green, blue) color space, expressingthese sets of color attribute values as respective color vectors, witheach color vector defined by a plurality of scalar values which arecoordinates of an orthogonal color space;

[0036] for each of a plurality of predetermined edge directions,generating a corresponding edge template as an array of respectivelypredetermined numeric values;

[0037] extracting an array of color vectors as respective color vectorsof an array of pixels having the object pixel as the center pixel ofthat array;

[0038] successively applying each of the edge templates to the array ofcolor vectors in a predetermined array processing operation, to deriveedge vectors respectively corresponding to the edge directions;

[0039] comparing the respective moduli of the derived edge vectors tofind the maximum modulus value, designating that maximum value as theedge strength of the object pixel and designating the edge directioncorresponding to an edge vector having that maximum modulus as being apossible edge direction for the object pixel; and,

[0040] judging whether the object pixel is located on an actual edgewhich is oriented in the possible edge direction, based upon comparingthe edge strength of the object pixel with respective values of edgestrength derived for pixels which are positioned immediately adjacent tothe object pixel and are on mutually opposite sides of the object pixelwith respect to the aforementioned possible edge direction.

[0041] The invention further provides an image recognition method foroperating on shape data expressing an original region image, (i.e., animage in which pixels are assigned respective labels indicative ofvarious image regions in which the pixels are located) to obtain shapedata expressing a region image in which specific small regions appearingin the original region image have been eliminated, with the methodcomprising repetitive execution of a series of steps of:

[0042] selectively determining respective regions of the original regionimage as constituting a set of small regions which are each to besubjected to a region combining operation;

[0043] selecting one of the set of small regions as a next small regionwhich is to be subjected to the region combining operation;

[0044] for each of respective regions which are disposed immediatelyadjacent to the next small region, calculating a length of commonboundary line with respect to the next small region, and determining oneof the immediately adjacent regions which has a maximum value of thelength of boundary line; and

[0045] combining the next small region with the adjacent region havingthe maximum length of common boundary line.

[0046] Data expressing a region image, to be processed by the method setout above, can be reliably derived by converting an edge image which hasbeen generated by the preceding method of the invention into a regionimage.

[0047] The above features of the invention will be more clearlyunderstood by referring to the following description of preferredembodiments of the invention

BRIEF DESCRIPTION OF THE DRAWINGS

[0048]FIG. 1 is a general system block diagram of a first embodiment ofan image recognition apparatus according to the present invention;

[0049]FIG. 2 is a conceptual diagram showing an example of actual colorattribute values of pixels in a color image, expressed in terms of anRGB color space;

[0050]FIG. 3 illustrates an RGB color space;

[0051]FIG. 4 is a diagram illustrating and edge image obtained as aresult of applying edge detection to a simplified color image;

[0052]FIG. 5 is a basic flow diagram of the operation of the firstembodiment;

[0053]FIGS. 6A to 6D are conceptual diagrams showing respective edgetemplates used with the first embodiment, and corresponding edgedirections;

[0054]FIG. 7 shows examples of a set of edge vectors;

[0055]FIG. 8 is a diagram illustrating how one of the edge vectors ofFIG. 7 defines the edge strength and possible edge direction for apixel;

[0056]FIGS. 9A to 9D are conceptual diagrams for illustrating how theedge strength of an object pixel is compared with the respective edgestrengths of pixels which are located adjacent thereto, on opposingsides with respect to an edge direction, for each of the possible edgedirections;

[0057]FIG. 10 is a diagram for use in describing how an edge image isobtained as a result of applying edge detection by the apparatus of thefirst embodiment to a simplified color image;

[0058]FIG. 11 is a flow diagram showing details of processing to deriveedge strength and possible edge direction information for each of thepixels of a color image in succession, with the first embodiment of theinvention;

[0059]FIG. 12 is a flow diagram showing details of processing, executedusing the edge strength and edge direction information derived in theflow diagram of FIG. 11, to determine those pixels of the color imagewhich are located on actual edges;

[0060]FIGS. 13, 14 are flow diagrams showing alternative forms of theprocessing executed in the flow diagrams of FIGS. 12 and 13respectively;

[0061]FIG. 15 is a general system block diagram of a second embodimentof an image recognition apparatus according to the present invention;

[0062]FIG. 16 is a basic flow diagram of the operation of the secondembodiment;

[0063]FIG. 17 is a diagram illustrating an orthogonal color spaceutilized with the second embodiment, in which the respective proportionsof color values of a pixel are expressed as coordinate values, ratherthan the color values themselves;

[0064]FIG. 18 is a diagram for use in describing how an edge image isobtained as a result of applying edge detection by the apparatus of thesecond embodiment to a simplified color image;

[0065]FIG. 19 is a flow diagram showing details of processing to deriveedge strength and possible edge direction information for each of thepixels of a color image in succession, with the second embodiment of theinvention;

[0066]FIG. 20 is a diagram illustrating an HSI color space utilized witha third embodiment of the invention;

[0067]FIG. 21 represents a simplified color image in which specificamounts of variation in color values occur within various regions of theimage;

[0068]FIG. 22 is a diagram showing an edge image which is obtained as aresult of applying edge detection by the apparatus of the thirdembodiment to the simplified color image of FIG. 21;

[0069]FIG. 23 is a flow diagram showing details of processing to deriveedge strength and possible edge direction information for each of thepixels of a color image in succession, with the third embodiment of theinvention;

[0070]FIG. 24 is a diagram illustrating a modified HSI color space, ofinverted conical form, utilized with a fourth embodiment of theinvention;

[0071]FIG. 25 is a diagram showing an edge image which is obtained as aresult of applying edge detection by the apparatus of the fourthembodiment to the simplified color image of FIG. 21;

[0072]FIG. 26 is a table of examples of sets of hue, saturation andintensity values which are derived by transforming the color values ofrespective regions of the color image represented in FIG. 21 intocorresponding values of a cylindrical (i.e., conventional) HSI colorspace, into an inverse-conical form of modified HSI color space, into adouble-conical modified HSI color space, and into a modified cylindricalHSI space respectively;

[0073]FIG. 27 is a partial flow diagram showing details of a first partof processing which is executed to derive edge strength and possibleedge direction information for each of the pixels of a color image insuccession, with the fourth embodiment of the invention;

[0074]FIG. 28 is a diagram illustrating a modified HSI color space, ofdouble-conical form, utilized with a fifth embodiment of the invention;

[0075]FIG. 29 is a diagram showing an edge image which is obtained as aresult of applying edge detection by the apparatus of the fifthembodiment to the simplified color image of FIG. 21;

[0076]FIG. 30 is a partial flow diagram showing details of a first partof processing which is executed to derive edge strength and possibleedge direction information for each of the pixels of a color image insuccession, with the fifth embodiment of the invention;

[0077]FIG. 31 is a graph of a saturation value modification functionwhich is utilized to transform color values into a modified cylindricalform of HSI color space, with a sixth embodiment of the invention;

[0078]FIG. 32 is a diagram illustrating the modified cylindrical HSIcolor space that is utilized with the sixth embodiment;

[0079]FIG. 33 is a diagram showing an edge image which is obtained as aresult of applying edge detection by the apparatus of the sixthembodiment to the simplified color image of FIG. 21;

[0080]FIG. 34 is a partial flow diagram showing details of a first partof processing which is executed to derive edge strength and possibleedge direction information for each of the pixels of a color image insuccession, with the sixth embodiment of the invention;

[0081]FIG. 35 is a general system block diagram of a seventh embodimentof an image recognition apparatus according to the present invention;

[0082]FIG. 36 is a conceptual diagram for illustrating the principles ofa region image;

[0083]FIG. 37 is a basic flow diagram of the operation of the seventhembodiment;

[0084]FIG. 38 is a diagram for use in describing a process ofeliminating specific small regions from a region image, performed by theseventh embodiment;

[0085]FIG. 39 is a diagram for use in describing a process ofeliminating specific small regions from a region image, performed by aneighth embodiment of the invention;

[0086]FIG. 40 is a basic flow diagram of the operation of the eighthembodiment;

[0087]FIG. 41 is a diagram for use in describing a process ofeliminating specific small regions from a region image, performed by aninth embodiment of the invention;

[0088]FIG. 42 is a basic flow diagram of the operation of the ninthembodiment;

[0089]FIG. 43 is a general system block diagram of a tenth embodiment ofan image recognition apparatus according to the present invention;

[0090]FIG. 44 is a basic flow diagram of the operation of the tenthembodiment;

[0091]FIG. 45 is a diagram for use in describing how specific smallregions are eliminated from a color image, by the apparatus of the tenthembodiment;

[0092]FIG. 46 is a diagram for illustrating the effect of the processingof the tenth embodiment in eliminating specific small regions from anedge image which has been derived by edge detection processing of anactual photograph; and

[0093]FIG. 47 shows a set of edge images which have been derived by aprior art type of image recognition apparatus, with hue, saturation andintensity edge images respectively obtained.

DESCRIPTION OF PREFERRED EMBODIMENTS

[0094] Embodiments of the present invention will be described in thefollowing, referring to the drawings. It should be noted that theinvention is not limited in its scope to these embodiments, and thatvarious other forms of these could be envisaged.

[0095] A first embodiment of an image recognition apparatus according tothe present invention will be described referring to FIG. 1. As usedherein in referring to embodiments of the invention, the term “imagerecognition” is used in the limited sense of signifying “processing thedata of an original color image to derive shape data, i.e., data of anedge image which expresses only the outlines of objects appearing in theoriginal color image”. The apparatus is formed of a color image datastorage section 1 which stores the data of a color image that is to besubjected to image recognition processing, an image recognitionprocessing section 2 which performs the image recognition processing ofthe color image data, and a shape data storage section 3 which storesshape data expressing an edge image, which have been derived by theimage recognition processing section 2.

[0096] The image recognition processing section 2 is made up of a colorvector data generating section 21, an edge template application section22, an edge strength and direction determining section 23 and an edgepixel determining section 24. The color vector data generating section21 generates respective color vectors for each of the pixels of thecolor image, with each color vector expressed as a plurality of scalarvalues which express the color attributes of the corresponding pixel andwhich are coordinates of an orthogonal color space having more than twodimensions. The edge template application section 22 processes the pixelvector data by utilizing edge templates as described hereinafter, togenerate edge vector data. Specifically, using four different edgetemplates with this embodiment which respectively correspond to fourdifferent orientation directions within the color image, a correspondingset of four edge vectors are derived for each of the pixels of the colorimage. The edge strength and direction determining section 23 operateson each of the pixels of the color image in succession, to determinewhether the pixel may be situated on an image, and if so, determines thedirection of orientation of that possible edge and its edge strength.The edge pixel determining section 24 operates on the information thusderived by the edge strength and direction determining section 23, todetermine those pixels which are actually judged to be edge pixels, andto thereby generate the shape data, i.e., data which express an edgeimage in which only the outlines of objects in the original color imageare represented.

[0097] As shown in the left side of FIG. 2, the image data stored in thecolor image data storage section 1 are assumed to be represented byrespective (x,y) coordinates of points in a 2-dimensional plane, i.e.,each pair of values (x,y) corresponds to one specific pixel. It is alsoassumed that the color attributes of each pixel are expressed as aposition in an RGB color space by three scalar values which arecoordinates of that space, i.e., as a set of r (red), g (green) and b(blue) values, as illustrated on the left side of FIG. 2. The functionof the color vector data generating section 21 is to express the colorattributes of each pixel of the color image as a plurality of scalarvalues which are coordinates of a vector in an orthogonal color space.If such a set of scalar values for each pixel is directly provided fromthe stored data of the color image data storage section 1 it will beunnecessary for the color vector data generating section 21 to performany actual processing. However if for example the data of the colorimage were stored in the color image data storage section 1 in someother form, e.g., with the color attributes of each pixel expressed as aset of polar coordinate, or with respective index values being storedfor the pixels, corresponding to respective sets of r, g,. b valueswithin a RGB table memory, then the color vector data generating section21 would perform all processing necessary to convert the data for eachpixel to a plurality of scalar values that are coordinates of an RGBorthogonal color space.

[0098] Moreover if desired, it would be possible for the color vectordata generating section 21 to be controlled to modify the relationshipsbetween the magnitudes of the r, g, b values of each pixel, to therebymodify the relative contributions of these to the magnitude of themodulus of a corresponding color vector.

[0099] It will be assumed that each of the r, g and b scalar values isformed of 8 bits, so that each value can be in the range 0 to 255. FIG.3 illustrates the RGB color space of these coordinates.

[0100] The data of a color image such as that shown in the upper part ofFIG. 4 will be assumed to be stored in the color image data storagesection 1, i.e., an image in which the objects are a street 40 and abuilding 41, in a ground area 42. The image recognition processingsection 2 applies edge detection to this image, to thereby obtain anedge image as shown in the lower part of FIG. 4, which is stored in theshape data storage section 3. The edge image is a bi-level image, i.e.,the black lines in the lower part of FIG. 4 correspond to pixels whichare situated along the edges of objects which appear in the originalcolor image, while the white portions correspond to pixels which do notcorrespond to edges. Basically, the edge detection that is executed bythe image recognition processing section 2 serves to detect the changebetween the color of the road 40 and the color of adjacent areas, andbetween the color of the building 41 and the color of adjacent areas,and to judge that each position where the amount of such change is largecorresponds to the position of an edge. The shapes of the street andbuilding are thereby detected as the shapes 50, 51 respectively, shownin the lower part of FIG. 4.

[0101]FIG. 5 is a flow diagram showing the basic features of theoperation of the first embodiment, which is executed as follows. Step10: Respective color vectors are derived for each of the pixels of thecolor image, with each color vector expressed as a combination of scalarvalues, which in this instance are constituted by the aforementioned r,g and b values of the pixel. The color vector of a pixel at position (x,y) of the color image, having the RGB scalar values r(x, y), g(x, y),b(x, y), is expressed by equation (1) below $\begin{matrix}{{{PV}\left( {x,y} \right)} = \begin{pmatrix}{r\left( {x,y} \right)} \\{g\left( {x,y} \right)} \\{b\left( {x,y} \right)}\end{pmatrix}} & (1)\end{matrix}$

[0102] Step 11: local multiplication and summing operations areperformed using the four edge templates h1, h2, h3, h4, to therebygenerate edge template data EV1, EV2, EV3, EV4 for each of the pixels ofthe color image. FIGS. 6A, 6B, 6C, 6D respectively show the four edgetemplates designated as h1, h2, h3, h4 which are utilized with thisembodiment. In FIG. 6A, h1 is an edge template corresponding to an edgethat is oriented in the left-right direction of the color image, andreturns a large value when this template is applied to an image positionwhere there is an edge that extends along the right-left direction.Similarly in FIG. 6B, h2 is an edge template for the lower left-upperright diagonal direction, in FIG. 6C h3 is an edge template for thetop-bottom direction, and in FIG. 6D h4 is an edge template for thelower right-upper left diagonal direction. As shown, each edge templatebasically consists of an array of numeric values which are divided intotwo non-zero sets of values, of mutually opposite sign, which arelocated symmetrically with respect to a line of zero values that isoriented in the edge direction corresponding to that edge template. Thevalues 0, 1, 2, −2 and −1 of the edge template h1 can be expressed asshown in equations (2) below. $\begin{matrix}\begin{matrix}{{{{h1}\left( {1,{- 1}} \right)} = 1},{{{h1}\left( {0,{- 1}} \right)} = 2},{{{h1}\left( {1,{- 1}} \right)} = 1}} \\{{{{h1}\left( {{- 1},0} \right)} = 0},{{{h1}\left( {{- 0},0} \right)} = 0},{{{h1}\left( {1,0} \right)} = 0}} \\{{{{h1}\left( {{- 1},{- 1}} \right)} = {- 1}},{{{h1}\left( {0,1} \right)} = {- 2}},{{{h1}\left( {1,1} \right)} = {- 1}}}\end{matrix} & (2)\end{matrix}$

[0103] The multiplication and summing processing that is applied betweenthe four both-direction edge templates and PV(x, y) is expressed byequations (3) below. $\begin{matrix}\begin{matrix}{{{EV1}\left( {x,y} \right)} = {\sum\limits_{k = {- 1}}^{1}\quad {\sum\limits_{l = 1}^{1}\quad {{{h1}\left( {k,l} \right)}P\quad {V\left( {{x + k},{y + l}} \right)}}}}} \\{{{EV2}\left( {x,y} \right)} = {\sum\limits_{k = {- 1}}^{1}\quad {\sum\limits_{l = 1}^{1}\quad {{{h2}\left( {k,l} \right)}P\quad {V\left( {{x + k},{y + l}} \right)}}}}} \\{{{EV3}\left( {x,y} \right)} = {\sum\limits_{k = {- 1}}^{1}\quad {\sum\limits_{l = 1}^{1}\quad {{{h3}\left( {k,l} \right)}P\quad {V\left( {{x + k},{y + l}} \right)}}}}} \\{{{EV4}\left( {x,y} \right)} = {\sum\limits_{k = {- 1}}^{1}\quad {\sum\limits_{l = 1}^{1}\quad {{{h4}\left( {k,l} \right)}P\quad {V\left( {{x + k},{y + l}} \right)}}}}}\end{matrix} & (3)\end{matrix}$

[0104] The above signifies that, designating the image position of thepixel that is currently being processed (i.e., the object pixel) as (x,y), a first edge vector EV1 (x, y) is obtained by multiplying the colorvector P(x−1, y−1) of the pixel which is located at the image position(x−1, y−1) by the scalar value that is specified for the (−1, −1)position in the edge template h1, i.e. by the value 1, multiplying thecolor vector P(x, y−1) of the pixel which is located at the imageposition (x, y−1) by the scalar value that is specified for the (0, −1)position in the edge template h1, i.e. by the value 2, and so on. Inthat way, the edge template h1 is applied to the color vector of theobject pixel and to the respective color vectors of eight pixels whichare located immediately adjacent to the object pixel in the color image.A set of nine vectors is thereby obtained, and the vector sum of theseis then calculated, to obtain a first edge vector EV1(x,y).

[0105] The above array multiplication and vector summing process isapplied using the other three edge templates h2, h3, h4 in the samemanner, to the object pixel and its adjacent pixels, to obtain the edgevectors EV2(x,y), EV3(x,y) and EV4(x,y) respectively corresponding tothese other three edge templates. The above process is executed for eachof the pixels of the color image in succession, as the object pixel.

[0106]FIG. 7 shows the four edge vectors that are obtained as a resultof applying the four edge templates of FIG. 6 to the color vector (r, g,b values 72, 183, 207 respectively) of the center pixel in the diagramat the right side of FIG. 2. EV1 is the edge vector corresponding to theleft-right direction, EV2 corresponds to the lower left-upper rightdiagonal direction, EV3 corresponds to the bottom-top direction, and EV4corresponds to the lower right-upper left diagonal direction.

[0107] Step 12: Using these edge vectors EV1, EV2, EV3, EV4, thestrength and orientation of an edge on which the object pixel may belocated are determined. That edge will be referred to in the followingas the “presumptive edge” obtained for the object pixel, which may ormay not be subsequently confirmed to be an actual edge as describedhereinafter. The strength of the presumptive edge obtained for theobject pixel having the image position (x, y), which is obtained as thevalue of the largest of the four moduli of the edge vectors EV1, EV2,EV3, EV4, will be designated as “MOD(x,y)”, and the direction of thatpresumptive edge will be designated as “DIR(x,y)”. That is to say,applying processing in accordance with equation (4) below, respectivevalues of strength of the presumptive edge, MOD(x,y) is obtained foreach of the pixels of the color image in succession, and the strengthvalues are stored temporarily.

mod (x, y)=max(|EV 1(x, y|), |EV 2(x, y)|, |EV 3(x, y)|, |EV 4(x,y)|)  (4)

[0108] If it is found when attempting to apply equation (4) that none ofthe moduli of the edge vectors obtained for a pixel exceeds all of theother edge vector moduli obtained for that pixel, then this may resultfrom all of the moduli of the edge vectors EV1(x,y), EV2(x,y), EV2(x,y),EV3(x,y) corresponding to the respective edge templates h1 to h4 beingof equal magnitude. In that case no possible edge direction is obtainedfor the object pixel, however the modulus value of the edge vectors isstored, as the edge strength value MOD obtained for that pixel, for usein subsequent processing.

[0109] Next, successively selecting each of the pixels (i.e., thosepixels for which a presumptive edge has been obtained) as the objectpixel and applying processing in accordance with equation (5) below, theorientation of the presumptive edge, designated in the following asDIR(x,y), is obtained for each of the pixels. That orientation is thedirection corresponding to the edge template whose application resultedin generation of the edge strength value MOD(x,y) for that pixel.Information specifying the obtained edge directions for the respectivepixels is temporarily stored. $\begin{matrix}{{{dir}\quad \left( {x,y} \right)} = \begin{matrix}\begin{matrix}\begin{matrix}{{{``{{Left}\text{-}{right}}"}\quad {if}\quad {{mod}\left( {x,\quad {\cdots \quad y}} \right)}} = {{{EV1}\left( {x,y} \right)}}} \\{{{``{{Lower}\quad {left}\text{-}{top}\quad {right}}"}\quad {if}\quad {{mod}\left( {x\quad,y} \right)}} = {{{EV2}\left( {x,y} \right)}}}\end{matrix} \\{{{``{{Bottom}\text{-}{top}}"}\quad {if}\quad {{mod}\left( {x\quad,y} \right)}} = {{{EV3}\left( {x,y} \right)}}}\end{matrix} \\{{{``{{Lower}\quad {r{ight}}\text{-}{top}\quad {left}}"}\quad {if}\quad {{mod}\left( {x\quad,y} \right)}} = {{{EV4}\left( {x,y} \right)}}}\end{matrix}} & (5)\end{matrix}$

[0110] For example, comparing the magnitudes of the respective moduli ofthe edge vectors shown in FIG. 7, the magnitude for EV3 is 437, which islarger than the magnitudes of each of the other edge vector moduli, sothat as shown in FIG. 8, the strength MOD of the presumptive edge ofthat pixel is obtained as 437. Also, since that edge strength valuecorresponds to the edge template h3 shown in FIG. 6, the edge directionof that presumptive edge is determined as being the bottom-top directionof the color image.

[0111] Step 13: the edge image data “EDGE” are generated, using apredetermined edge strength threshold value t, the respectivepresumptive edge strength values “MOD” obtained for the pixels of thecolor image and the respective edge directions “DIR” obtained for thepixels, in the manner indicated by equation (6) below $\begin{matrix}{{{edge}\left( {x,y} \right)} = {{``{edge}"}\quad {if}\text{:}\quad \left( {{{{{mod}\left( {x,y} \right)} \geq t}\&}\left( {\left( {{{dir}\left( {x,y} \right)} = {{{{{{{{``{{left}\text{-}{right}}"}\quad {direction}}\&}{{mod}\left( {x,y} \right)}} > {{mod}\left( {x,{y - 1}} \right)}}\&}\quad {{mod}\left( {x,y} \right)}} > {{mod}\left( {x,{y + 1}} \right)}}} \right),{{or}\quad {if}\quad \left( {{{{{mod}\left( {x,y} \right)} \geq t}\&}\left( {\left( {{{dir}\left( {x,y} \right)} = {{{{{{{{``{{lower}\quad {left}\text{-}{top}\quad {right}}"}\quad {direction}}\&}{{mod}\left( {x,y} \right)}} > {{mod}\left( {{x - 1},{y - 1}} \right)}}\&}{{mod}\left( {x,y} \right)}} > {{mod}\left( {{x + 1},{y + 1}} \right)}}} \right),{{or}\quad {if}\quad \left( {{{{{mod}\left( {x,y} \right)} \geq t}\&}\left( {\left( {{{dir}\left( {x,y} \right)} = {{{{{{{{``{{bottom}\text{-}{top}}"}\quad {direction}}\&}{{mod}\left( {x,y} \right)}} > {{mod}\left( {{x - 1},y} \right)}}\&}{{mod}\left( {x,y} \right)}} > {{mod}\left( {{x + 1},y} \right)}}} \right)\quad,{{or}\quad {if}\quad \left( {{{{{mod}\left( {x,y} \right)} \geq t}\&}\left( {{\left( {{{dir}\left( {x,y} \right)} = {{{{{{{{``{{left}\text{-}{right}}"}\quad {direction}}\&}{{mod}\left( {x,y} \right)}} > {{mod}\left( {x,{y - 1}} \right)}}\&}{{mod}\left( {x,y} \right)}} > {{mod}\left( {x,{y + 1}} \right)}}} \right).{otherwise}},\quad {{{edge}\left( {x,y} \right)} \neq {``{edge}"}}} \right.} \right.}} \right.} \right.}} \right.} \right.}} \right.} \right.}} & (6)\end{matrix}$

[0112] That is to say, the pixels for which respective presumptive edges(i.e., possible edge directions) have been derived are successivelyselected as the object pixel, with the threshold value t, edge strengthMOD(x,y) and edge direction DIR(x,y) of the object pixel being used tomake a decision as to whether or not the object pixel actually is anedge pixel. With equation (6), if a pixel has an edge strength that ishigher than t, and the relationship between that pixel and the adjacentpixels satisfies one of the four patterns which are shown in FIGS. 9A to9C, then it is judged that this is an edge pixel.

[0113] More specifically, numeral 200 in FIG. 9A designates an array ofsix pixels of the color image, centered on a pixel 202 which iscurrently being processed as the object pixel. The designations “weak”,“strong” indicate the relationships between the respective values ofstrength that have been previously obtained for the pixels as describedabove. In FIG. 9A, it is assumed that the edge strength MOD obtained forpixel 202 is the edge vector modulus that is obtained by using the edgetemplate h1 shown in FIG. 6, i.e., EV1(x,y) in equation (3) describedabove, and hence the orientation DIR of the presumptive edgecorresponding to pixel 202 is the left-right direction of the colorimage, i.e. a presumptive edge has been derived for pixel 202 as astraight line of undefined length which passes through that pixel and isoriented in the horizontal direction of FIG. 9A. It is further assumedin FIG. 9A that the respective values of edge strength derived for thetwo pixels 201, 203 which are immediately adjacent to the object pixel202 and disposed on opposing sides of the presumptive edge derived forthe object pixel 202 are both less than the value of strength that hasbeen derived for the presumptive edge of the object pixel 202. In thatcondition, if that edge strength value obtained for the object pixel 202exceeds the edge threshold value t, then it is judged that pixel 202 islocated on an actual edge within the color image.

[0114] Similarly in FIG. 9B, the presumptive edge that has been derivedfor the object pixel 205 is a line extending through the pixel 205,oriented in the lower left-upper right diagonal direction of the colorimage, and the respective values of strength derived for the two pixels204, 206 which are immediately adjacent to the object pixel 205 anddisposed on opposing sides of the presumptive edge derived for theobject pixel 205 are both less than the value of strength that has beenderived for the presumptive edge of the object pixel 205. Thus in thesame way as for the example of FIG. 9A, assuming that the edge strengthvalue obtained for the object pixel 205 exceeds the edge threshold valuet, it will be judged that pixel 205 is located on an actual edge,oriented diagonally as shown in FIG. 9B within the color image. In asimilar way, it will be judged that the object pixel is located on anactual vertically oriented edge if the pattern condition of FIG. 9C issatisfied, or on an actual edge which is oriented along the lowerright-upper left diagonal direction, if the pattern condition of FIG. 9Dis satisfied.

[0115] As can be understood from the above description, the effect ofapplying one of the edge templates shown in FIGS. 6A to 6D to an arrayof color vectors centered on an object pixel is to obtain (as an edgevector) the vector difference between the weighted vector sum of thecolor vectors of a first set of pixels which are located on one side ofthe object pixel with respect to the edge direction of that template(i.e., whose vectors are multiplied by 1, 2, and 1 respectively) and theweighted vector sum of the color vectors of a second set of pixels whichare located on the opposite side of the object pixel (i.e., whosevectors are multiplied by −1, −2 and −1, respectively). It will befurther understood that the invention is not limited to theconfigurations of edge templates utilized with this embodiment.

[0116]FIG. 11 is a flow diagram showing details of the processingperformed in steps 10 to 12 of FIG. 5, to derive the edge vectors andthe edge strength “mod” and edge direction “dir” information for thepixels of the color image that is to be processed. The sequence of steps1001 to 1010 of FIG. 11 are repetitively executed for each of the pixelsof the color image in succession, i.e., with the pixels beingsuccessively selected as the object pixel for which mod and dirinformation are to be derived. In step 1002, a plurality of scalarvalues expressing a color vector in that orthogonal RGB color space forthe object pixel are read out from the color image data storage section1 (i.e., the r, g and b values for the object pixel) as are also therespective sets of RGB values expressing the color vectors of the groupof eight pixels which are immediately adjacent to the object pixel andsurround the object pixel. In step 1003 that array of nine color vectorsis successively multiplied by each of the arrays of values whichconstitute the edge templates h1, h2, h3 and h4, in the manner describedhereinabove, with the respective vector sums of the results beingobtained as the edge vectors EV1, EV2, EV3 and EV4. In step 1004, themoduli of these edge vectors are obtained and are compared, to find ifone of these is greater than each of the other three. If this conditionis met, as determined in step 1006, then that largest value of modulusis temporarily stored in an internal memory (not shown in the drawings)as the edge strength MOD(x,y) of the object pixel, together withinformation indicating the direction corresponding to that largest edgevector as the orientation DIR(x,y) of the object pixel.

[0117] However if the condition whereby one of moduli of EV1, EV2, EV3and EV4 is greater than each of the other three is not satisfied thenstep 1005 is executed to judge whether all of the vector moduli have thesame value. If that condition is found, then no direction can beobtained as DIR(x,y) for the object pixel, and only that modulus valueis stored as the edge strength MOD(x,y) for the object pixel, in step1007. If that condition is not found (i.e., two or three of the vectormoduli have the same value, which is greater than that of the remainingone(s)) then the modulus of an arbitrarily selected one of the edgevectors which have the largest value is selected as the edge strengthMOD(x,y) of the object pixel, while the orientation of the edge templatecorresponding to that selected edge vector is stored as the edgedirection DIR(x,y) of the object pixel, in step 1008.

[0118]FIG. 12 is a flow diagram showing details of the processingperformed in step 13 of FIG. 5, to derive the shape data which are to beoutput and stored in the image recognition processing section 2, i.e.,to find each of the pixels which is actually located on an edge withinthe color image, and the corresponding edge direction. The sequence ofsteps 1011 to 1017 of FIG. 12 is successively applied to each of thepixels of the color image for which edge direction information DIR hasbeen derived and temporarily stored, together with corresponding edgestrength information MOD, as described above. In steps 1011, 1012 thenext pixel to which this processing is to be applied as the object pixelis selected, and the edge strength MOD(x,y) and edge direction DIR(x,y)information for that object pixel are read out. If it is judged in step1013 that the value of MOD(x,y) is greater than or equal to the edgethreshold value t, then step 1004 is executed, to read out therespective values of edge strength of the two pixels which are locatedimmediately adjacent to the object pixel and on mutually opposite sidesof the presumptive edge that has been detected for the object pixel.

[0119] Next, in step 1015, the three values of edge strength arecompared, to determine if the edge strength MOD(x,y) of the object pixelis greater than the edge strengths of both these adjacent pixels. If so,then the pixel which corresponds in position to the object pixel withinthe image expressed by the shape data (i.e., the edge image) isspecified as being located on an actual edge, which is oriented in thedirection DIR(x,y). In that way, the shape data expressing the edgeimage are successively derived as binary values which indicate, for eachpixel of the color image, whether or not that pixel is located on anedge.

[0120] It can thus be understood that with the above processing, a pixelof the color image, when processed as the object pixel, will be judgedto be located on an actual edge within the color image if it satisfiesthe conditions:

[0121] (a) an edge direction DIR, and also a value of edge strength MODthat exceeds the edge threshold value t, have been obtained for thatobject pixel, and

[0122] (b) the edge strength MOD of that object pixel is greater thaneach of the respective edge strengths of the two pixels which arelocated immediately adjacent to the object pixel and are on mutuallyopposite sides of a presumptive edge (i.e., a line which is oriented indirection DIR, passing through that pixel) that has been obtained forthe object pixel.

[0123] With the operation of FIG. 11, FIG. 12 described above, in theevent that it is found in step 1005 that there are a plurality of edgevectors having the same magnitude of modulus, which is greater than thatof the remaining vector(s), for example if the moduli of EV1, EV2 areidentical and each are larger than the respective moduli of EV3, EV4,then the edge direction corresponding to an arbitrarily selected one ofthe largest edge vectors is selected to be used as the edge directionDIR of the object pixel, in step 1008. However various other procedurescould be used when such a condition occurs. An alternative procedure isillustrated in the flow diagrams of FIGS. 13, 14. In step 1008 b of FIG.13, the respective edge template directions corresponding to each of theedge vectors having the largest moduli are all stored as candidates forthe edge direction DIR of the object pixel, together with the maximumedge vector modulus value as the edge strength MOD. In that case, asshown in FIG. 14, if the pixel which has been selected as the objectpixel in step 1011 is found to have a plurality of correspondingcandidate edge directions DIR stored, then the information specifyingthese different directions are successively read out in repetitions of astep 1012 b. That is to say, the processing of steps 1012 b to 1015 isrepetitively executed for each of these directions until either it isfound that the condition of step 1015 is satisfied (the pixel is judgedto be on an actual edge) or all of the candidate edge directions forthat pixel have been tried, as judged in step 1018. In other respects,the processing shown is identical to that of FIGS. 11, 12 describedabove.

[0124] A specific example will be described in the following. The upperpart of FIG. 10 shows data of a color image, expressed as coordinates ofan RGB color space, representing a simplified aerial photograph which isto be subjected to image recognition. The image is identical to that ofFIG. 4, containing a street, ground, and a building, with the buildingroof and first and second side faces of the building appearing in theimage. Respective RGB values for each of these are assumed to be asindicated in the drawing. For example it is assumed that each of thepixels representing the ground surface have the r, g and b values 195,95 and 0 respectively. By applying the first embodiment of the inventionto this image to process the data of the color image in the mannerdescribed above, bi-level shape data are obtained from the imagerecognition processing section 2 and stored in the shape data storagesection 3, with the shape data expressing the outlines of the street andthe building roof and side faces in the form of edges, as shown in thelower part of FIG. 10, i.e., with the shape of the street formed as twoedges 50, and the shape of the building roof and side faces being formedas the set of edges 51.

[0125] As described above, with the present invention, pixel vector dataare generated as combinations of pluralities of scalar valuesconstituting pixel values, and edge detection is performed by operatingon these pluralities of scalar values. With prior art types of edgedetection which operate only upon values of intensity, even if theoutlines of a body exist within an image but the outlines are not in theform of variations in intensity, then edge detection cannot be achievedfor that body. However with the present invention, in such a condition,edge detection becomes possible.

[0126] Furthermore, by applying edge templates to pixel vector data,edge directions can be obtained easily and reliably. If the direction ofan edge is known, then it becomes possible to form that edge as acontinuous line (as expressed in the shape image that is generated) evenif all of the pixels corresponding to that edge are not detected. Thatis to say, if the direction of an edge can be reliably obtained on thebasis of a part of the pixels of that edge, then interpolation of theremaining pixels can readily be performed, to thereby eliminate anybreaks in the continuity of the edge. For that reason, the basic featureof the present invention whereby it is possible not only to detect thestrengths of edges, but also to reliably estimate their directions, ishighly important.

[0127] A second embodiment of an image recognition apparatus accordingto the present invention is shown in the general system block diagram ofFIG. 15. Here, sections having similar functions to those of theapparatus of the first embodiment shown in FIG. 1 are designated byidentical reference numerals to those of FIG. 1. In the apparatus ofFIG. 15, the color vector data generating section 121 performs a similarfunction to that of the color vector data generating section 21 of thefirst embodiment, but in addition receives control parameter adjustmentdata, supplied from an external source as described hereinafter. Inaddition, the apparatus of FIG. 15 further includes a color spacecoordinates conversion section 25 is for performing color spacecoordinate transform processing. The data stored in the image datastorage section 1, which in the same way as described for the firstembodiment will be assumed to directly represent a color image as setsof r, g, b values that are coordinates of an RGB color space, aretransformed to cords of a different orthogonal color space,specifically, a color space in which chrominance and intensity valuesare mutually separated. Color vectors are then generated for each of thepixels data by the color vector data generating section 121 using theresults of the transform operation.

[0128]FIG. 16 is a flow diagram showing the basic features of theoperation of the second embodiment.

[0129] Steps 11, 12 and 13 of this flow diagram are identical to thoseof the basic flow diagram of the first embodiment shown in FIG. 5. Step10 of this flow diagram differs from that of the first embodiment inthat color vector modulus adjustment can be performed, as describedhereinafter. A new step 20 is executed as follows.

[0130] Step 20: the color attribute data of each pixel are transformedfrom the RGB color space to coordinates of the color space shown in FIG.17. Specifically, each set of pixel values r(x, y), g(x, y), b(x, y) isoperated on, using equation (7), to obtain a corresponding set ofcoordinates c1(x, y), c2(x, y), c3(x, y). Here, c1 expresses a form ofintensity value for the pixel, i.e., as the average of the r, g and bvalues of the pixel, c2 expresses the proportion of the red component ofthat pixel in relation to the total of the red, green and blue valuesfor that pel, and C3 similarly expresses the proportion of greencomponent of that pixel in relation to the total of the red, green andblue values of that pixel. $\begin{matrix}\begin{matrix}{{{c1}\left( {x,y} \right)} = \frac{{r\left( {x,y} \right)} + {g\left( {x,y} \right)} + {b\left( {x,y} \right)}}{3}} \\{{{c2}\left( {x,y} \right)} = {\frac{r\left( {x,y} \right)}{{r\left( {x,y} \right)} + {g\left( {x,y} \right)} + {b\left( {x,y} \right)}} \cdot {max\_ value}}} \\{{{c3}\left( {x,y} \right)} = {\frac{g\left( {x,y} \right)}{{r\left( {x,y} \right)} + {g\left( {x,y} \right)} + {b\left( {x,y} \right)}} \cdot {max\_ value}}}\end{matrix} & (7)\end{matrix}$

[0131] As can be understood from the above equation and FIG. 17, thecolor attributes of a pixel having the maximum r value (i.e., 255) andzero g and b values, in the RGB color space, are expressed as a positionwithin the color space of FIG. 17 which has the c1, c2, c3 coordinates(255/3, 255, 0). This is the point designated as “red” in FIG. 17.Similarly, points which correspond to the “maximum blue component, zerored and green components” and “maximum green component, zero red andblue components” conditions within the RGB color space are respectivelyindicated as the “blue” and “green” points in FIG. 17.

[0132] Step 10: the pixel vector data PV are generated from the pixelvalues. Pixel vector data are generated for each pixel based on acombination of the attribute values of the pixel. A vector data setPV(x, y) is generated for each of the pixels, by applying equation (8)below to the pixel values c1(x, y), c2(x, y), c3(x, y). By adjusting theparameters a1, a2 and a3 of equation (8), through input of controlparameter adjustment data to the color vector data generating section121, it is possible to determine whether the edge detection will bebased mainly on the c1 values, the c2 values, or on the c3 values, i.e.,the relative contributions made by the c1, c2 and c3 coordinates of acolor vector to the magnitude of the modulus of the color vector can beadjusted by altering the values of the control parameters a1, a2 and a3.The resultant color vector is expressed as follows. $\begin{matrix}{{P\quad {V\left( {x,y} \right)}} = \begin{pmatrix}{{a1} \cdot {{c1}\left( {x,y} \right)}} \\{{a2} \cdot {{c2}\left( {x,y} \right)}} \\{{a3} \cdot {{c3}\left( {x,y} \right)}}\end{pmatrix}} & (8)\end{matrix}$

[0133]FIG. 19 is a flow diagram showing the processing executed withthis embodiment to derive the candidate edge strength values (MOD) andedge directions (DIR) for the pixels of the color image. As shown, thisdiffers from the corresponding diagram of FIG. 11 of the firstembodiment only with respect to the steps 1002 a, 1002 b which replacestep 1002 of FIG. 11, for deriving the color vectors as sets ofcoordinates expressing respective positions within the color space ofFIG. 17.

[0134] A specific example will be described in the following. The upperpart of FIG. 18 shows data of a color image representing a simplifiedaerial photograph which is to be subjected to image recognition.Examples of the r, g and b values for various regions of the colorimage, and the corresponding sets of c1, c2, c3 values which express thecolor attributes of these regions as positions in the color space ofFIG. 17 are also indicated in the drawing. As described above therespective sets of r, g and b values of the pixels, for the RGB colorspace, are converted to corresponding sets of c1, c2, c3 coordinates,the values of the control parameters a1, a2, a3 are set in accordancewith the characteristics of the color image (for example if required,such that differences in respective intensity values between adjacentregions will have a relatively large effect upon the differences betweenmagnitudes of corresponding color vectors as described hereinabove), andrespective color vectors for the pixels of the color image, expressed indetection is then performed, to obtain the shape of the street and thebuilding as designated by numerals 50 and 51 respectively in the lowerpart of FIG. 18.

[0135] As described above, with this embodiment, respective colorvectors for the pixels of the color image are derived by transformprocessing of the stored image data into coordinates of a color spacewhich is more appropriate for edge detection processing than theoriginal RGB color space. That is to say, the image data are subject toconversion to color space coordinates whereby the edge detectionprocessing can be adjusted (i.e., by altering the relative values of thecontrol parameters) such as to match the edge detection processing tothe particular characteristics of the image that is to be subjected toimage recognition processing. For example, if differences betweenvarious regions of the image are primarily gray-scale variations, i.e.,variations in intensity rather than in chrominance, then this fact canreadily be judged beforehand by a human operator, and the controlparameter values adjusted such as to emphasize the effects of variationsin intensity values upon the edge detection process.

[0136] A third embodiment of an image recognition apparatus according tothe present invention will be described. The apparatus configuration isidentical to that of the second embodiment (shown in FIG. 15).

[0137] The basic operation sequence of this embodiment is similar tothat of the second embodiment, shown in FIG. 16. However with the thirdembodiment, the transform is performed from an RGB color space to an HSIcolor space, instead of the color space of FIG. 17. That is to say,steps 11, 12 and 13 are identical to those of the first embodiment,however step 20 is performed as follows. Step 20: each pixel value istransformed from the RGB color space to the coordinates of thecylindrical color space shown in FIG. 20. Each set of pixel values r(x,y), g(x, y), b(x, y) is operated on, using equation (9), to obtain acorresponding set of hue, saturation and intensity values as h(x, y),s(x, y) and i(x, y) respectively of the HSI color space of FIG. 20. Inthis case, the gray-scale values, i.e. values of intensity extendingfrom black (as value 0) to white (as maximum value), are plotted alongthe vertical axis of the cylindrical coordinate system shown in the leftside of FIG. 20. $\begin{matrix}\begin{matrix}{{i\quad \max} = {\max \left( {r,g,b} \right)}} \\{{i\quad \min} = {\min \left( {r,g,b} \right)}} \\{i = \frac{{i\quad \max} + {i\quad \min}}{2}} \\{s = \left\{ \begin{matrix}0 & {{{if}\quad i\quad \max} = {i\quad \min}} \\{\frac{{i\quad \max} - {i\quad \min}}{{i\quad \max} + {i\quad \min}}{max\_ value}} & {{{if}\quad i} \leq \frac{max\_ value}{2}} \\{\frac{{i\quad \max} - {i\quad \min}}{{max\_ value} - {i\quad \max} - {i\quad \min}} \cdot {max\_ value}} & {{{if}\quad i} > \frac{max\_ value}{2}}\end{matrix} \right.} \\{{r1} = \frac{{i\quad \max} - r}{{i\quad \max} - {i\quad \min}}} \\{{g1} = \frac{{i\quad \max} - g}{{i\quad \max} - {i\quad \min}}} \\{{b1} = \frac{{i\quad \max} - b}{{i\quad \max} - {i\quad \min}}} \\{{r1} = \frac{{i\quad \max} - r}{{i\quad \max} - {i\quad \min}}} \\{{g1} = \frac{{i\quad \max} - g}{{i\quad \max} - {i\quad \min}}} \\{{b1} = \frac{{i\quad \max} - b}{{i\quad \max} - {i\quad \min}}} \\{h = \left\{ \begin{matrix}{undefined} & {{{if}\quad i\quad \max} = {i\quad \min}} \\\frac{\left( {{b1} - {g1}} \right)\pi}{3} & {{{if}\quad r} = {i\quad \max}} \\\frac{\left( {2 + {r1} - {b1}} \right)\pi}{3} & {{{if}\quad g} = {i\quad \max}} \\\frac{\left( {4 + {g1} - {r1}} \right)\pi}{3} & {{{if}\quad b} = {i\quad \max}}\end{matrix} \right.}\end{matrix} & (9)\end{matrix}$

[0138] The saturation value expresses the depth of a color, andcorresponds to a distance extending radially from the center of thecoordinate system shown in the right side of FIG. 20. The hue valuecorresponds to an angle in the coordinate system shown on the right sideof FIG. 20. For example when this angle is zero degrees, thiscorresponds to the color red, while an angle of ⅔π radians correspondsto blue.

[0139] It should be noted that there are various models for performingthe transform from an RGB to an HSI color space, and that the presentinvention is not limited to use of equation (9) for that purpose. Withequation (9) the range of values of each of r, g, b, i, and s is from 0to the maximum value (i.e., 255 in the case of 8-bit data values),designated as “max_value”. The range of values of h is from 0 to 2πradians. For simplicity, the image position coordinates (x, y) have beenomitted from the equation.

[0140] With this embodiment, step 10 of the flow diagram of FIG. 16 isexecuted as follows. Using equation (10) below, color vectors PV(x, y)are generated for each of the pixels, from the hue, saturation andintensity values h(x, y), s(x, y), i(x, y) of each pixel.$\begin{matrix}{{P\quad {V\left( {x,y} \right)}} = \begin{pmatrix}{a \cdot {s\left( {x,y} \right)} \cdot {\cos \left( {h\left( {x,y} \right)} \right)}} \\{a \cdot {s\left( {x,y} \right)} \cdot {\sin \left( {h\left( {x,y} \right)} \right)}} \\{i\left( {x,y} \right)}\end{pmatrix}} & (10)\end{matrix}$

[0141] Here each color vector PV is generated by converting the portionsh(x, y), s(x, y) that are expressed in polar coordinates to a linearcoordinate system. By adjusting the value of the control parameter “a”,it becomes possible for example to place emphasis on the intensityvalues, in the edge detection processing. For example if the value ofthe parameter “a” is made equal to 1, then edge detection processingwill be performed placing equal emphasis on all of the values in the HSIspace, while if the value of the parameter a is made less than 1, thenedge detection processing will be performed placing greater emphasis onintensity values.

[0142] That is to say, the relative contribution of the intensitycomponent of the color attributes of a pixel to the magnitude of themodulus of the color vector of that pixel will increase in accordancewith decreases in the value of the control parameter “a”.

[0143] The operation of this embodiment for generating respective colorvectors corresponding to the pixels of the color image is shown in moredetail in the flow diagram of FIG. 23. This differs from thecorresponding flow diagram of FIG. 11 for the first embodiment in thatthe step 1002 of the first embodiment, for deriving the array of colorvectors PV which are to be operated on using the edge templates inequation (2) as described above to obtain the edge vectors EV1(x,y) toEV2(x,), is replaced by a series of three steps, 1002 a, 1002 c and 1002d.

[0144] In the first of these, step 1002 a, the respective sets of r, g,b values for the object pixel and its eight adjacent surrounding pixelsare obtained from the image data storage section 1, and in step 1002 ceach of these sets of r, g, b values of the RGB color space is convertedto a corresponding set of h, s, i values of the cylindrical HSI colorspace shown in FIG. 20. In step 1002 d, each of these sets is convertedto a corresponding set of three linear coordinates, i.e., of anorthogonal color space, using the trigonometric operation describedabove, to thereby express the hue and saturation information of eachpixel in terms of linear coordinates instead of polar coordinates, whileeach of the resultant s. cos h and s. sin h values is multiplied by thecontrol parameter “a”, as indicated by equation (10).

[0145] A specific example will be described in the following. FIG. 21shows data of a color image representing a simplified aerial photographwhich is to be subjected to image recognition. As opposed to the imageof the upper part of FIG. 10, it is assumed with the image of FIG. 21that there are ranges of variation of pixel values, as would occur inthe case of an actual aerial photograph. Thus in each of the regions ofthe color image, rather than all of the RGB values of that region beingidentical, there is a certain degree of scattering of these pixelvalues.

[0146] As described above, the color attributes of the pixels of thecolor image are converted from RGB to HSI color space coordinates, whichare then converted to respective coordinates of an orthogonal system byapplying equation (10) above, to thereby obtain respective color vectorscorresponding to the pixels, and edge detection processing then appliedto the color vectors in the same manner as described for the firstembodiment. The result of applying this processing to the image shown inFIG. 21 is illustrated in FIG. 22. As shown, the shapes of the streetand the building have been extracted from the original image, asindicated by numerals 52 and 53 respectively. Due to the scattering ofpixel values in the original color image, some level of noise will arisein the edge detection process, so that as shown in FIG. 21, somediscontinuities occur in the outlines of the street and the building.

[0147] Thus with this embodiment of the present invention, pixel vectordata are generated after having converted pixel values which have beenstored as coordinates of a certain color space into the coordinates ofan HSI color space, which are then converted to linear coordinates of acolor space in which the luminance and chrominance informationcorrespond to respectively different coordinates. This simplifies edgedetection, since the overall hue, saturation and intensitycharacteristics of a color image can be readily judged by a humanoperator, and the value of the control parameter “a” can thereby be setappropriately by the operator, to enable effective edge detection to beachieved.

[0148] A fourth embodiment of an image recognition apparatus will bedescribed. The configuration is basically similar to that of the secondembodiment (shown in FIG. 15).

[0149] The operation sequence of this embodiment is similar to that ofthe second embodiment, shown in the flow diagram of FIG. 16, with steps11, 12 and 13 being identical to those of the first embodiment. Thecontents of step 20 of FIG. 16, with the fourth embodiment, differ fromthose of the second embodiment and are as follows.

[0150] Step 20: the pixel values are transformed from the RGB colorspace to the coordinates of the cylindrical HSI color space shown inFIG. 20, using equation (9) as described hereinabove for the thirdembodiment. Equation (11) is then applied to transform the respectivesets of h, s, i values obtained for each of the pixels of the colorimage pixel to the coordinates of a color space of the inverted conicalform shown in FIG. 24, i.e., to coordinates h′, s′, i′ of a modifiedform of HSI color space. $\begin{matrix}\begin{matrix}{h^{\prime} = {\left( {x,y} \right) = {h\left( {x,y} \right)}}} \\{s^{\prime} = {\left( {x,y} \right) = {\frac{i\left( {x,y} \right)}{max\_ value} \cdot {s\left( {x,y} \right)}}}} \\{{i^{\prime}\left( {x,y} \right)} = {i\left( {x,y} \right)}}\end{matrix} & (11)\end{matrix}$

[0151] Thus, the color space transform operation is performed byapplying equation (11) above to convert each h(x, y), s(x, y), i(x, y)set of values, for the pixel located at position (x, y) of the colorimage, to a set of h′(x, y), s′(x, y), i′(x, y) values respectively.This transform does not produce any change between h(x, y) and h′(x, y),or between i(x, y) and i′(x, y), however as the value of i(x, y) becomessmaller, the value of s′(x, y) is accordingly reduced.

[0152] With this embodiment, the contents of step 1010 of the flowdiagram of FIG. 16 are as follows. Respective color vectors aregenerated for each of the pixels, with the vectors expressed asrespective sets of linear coordinates of an orthogonal color space, byapplying equation (12) below to the set of polar coordinates h′(x, y),s′(x, y), i′(x, y) that have been derived for the pixel by applyingequation (11) $\begin{matrix}{{{PV}\left( {x,y} \right)} = \begin{pmatrix}{a \cdot {s\left( {x,y} \right)} \cdot {\cos \left( {h\left( {x,y} \right)} \right)}} \\{a \cdot {s\left( {x,y} \right)} \cdot {\sin \left( {h\left( {x,y} \right)} \right)}} \\{i\left( {x,y} \right)}\end{pmatrix}} & (12)\end{matrix}$

[0153] Thus, each color vector is generated by converting the portionsh′(x, y), s′(x, y) of the h′, s′, i′ information for each pixel , i.e.,the values that are expressed in polar coordinates, to a linearcoordinate system. By adjusting the value of the parameter “a,” the formof emphasis of the edge detection processing can be altered, i.e., therelative contribution of the intensity component of the color attributesof each pixel to the magnitude of the modulus of the color vector thatis derived for the pixel can be modified, by adjusting the value of thecontrol parameter “a”, so that it becomes possible to place emphasis onvariations in intensity between adjacent regions, in the edge detectionprocessing. For example if the value of the parameter “a” is made equalto 1, then edge detection processing will be performed placing equalemphasis on all of the hue, saturation and intensity values, while ifthe value of the parameter “a” is made less than 1, then edge detectionprocessing will be performed placing greater emphasis on intensityvalues.

[0154] The operation of this embodiment for generating respective colorvectors corresponding to the pixels of the color image is shown in thepartial flow diagram of FIG. 27. This differs from the correspondingflow diagram of FIG. 11 for the first embodiment in that the step 1002of the first embodiment, for deriving the array of color vectors PVwhich are to be operated on by applying the edge templates in equation(2) as described above to obtain the edge vectors EV1(x,y) to EV2(x,),is replaced by a series of four steps, 1002 a, 1002 c, 1002 e and 1002f. In step 1002 a, the respective sets of r, g, b values for the objectpixel and its eight adjacent surrounding pixels are obtained from thecolor image data storage section 1, and in step 1002 c each of thesesets of r, g, b values of the RGB color space is converted to acorresponding set of h, s, i values of the cylindrical-shape HSI colorspace shown in FIG. 20. In step 1002 e, each of these sets of h, s, ivalues is converted to a corresponding set of h′, s′, i′ values of theinverted-conical H′S′I′ color space. In step 1002 f, each of these setsis converted to a corresponding set of three linear coordinates, i.e.,of an orthogonal color space, while each of the resultant s′. cos h′ ands′. sin h′ values is multiplied by the control parameter “a”, asindicated by equation (12).

[0155] The remaining steps of this flow diagram, which are omitted fromFIG. 27, are identical to steps 1003 to 1010 of FIG. 11.

[0156] A specific example will be described in the following. In thesame way as for the third embodiment, it will be assumed that thesimplified aerial photograph of FIG. 21 is the image that is to besubjected to recognition processing.

[0157] As described above, the RGB values of the pixels are firstconverted to HSI values of the cylindrical color space of FIG. 20, andthese are then transformed to H′S′I′ form, as coordinates of theinverted-conical color space shown in FIG. 24. The first and secondcolumns of values in the table of FIG. 26 show the relationship betweenrespective HSI values for each of the regions, and the correspondingH′S′I′ values resulting from the transform. In the case of the transforminto the HSI space, the lower the values of intensity become, thegreater will become the degree of scattering of the values ofsaturation. This is a characteristic feature of the transform from RGBto the HSI space. For example, if all of the RGB values of a pixel aresmall, signifying that the intensity is low, then a change of 1 in anyof the RGB values will result in an abrupt change in the correspondingsaturation value. Thus, since sudden changes in color will occur atpositions where such abrupt variations in the saturation values occur,edges may be erroneously detected even at positions where there is noactual border of any of the objects which are to be recognized. Howeverin the case of a transform into H′S′I′ values of the inverse-conical HSIspace, the lower the value of intensity of the pixels, the smaller willbecome the value of s′, so that the scattering of the values of s′ issuppressed. As a result, random abrupt changes in the magnitudes of themoduli of the color vectors which are derived by applying equation (12)can be eliminated, enabling greater accuracy of edge detection.

[0158]FIG. 25 shows the image recognition processing results which areobtained when this embodiment is applied to edge detection of the colorimage represented in FIG. 21. The building face 1 and building face 2 inthe image of FIG. 21 are each regions of low values of intensity, sothat the noise level for these regions, due to erroneous detection ofspurious edges, could be expected to be high. However as shown in FIG.25, such noise is substantially suppressed, with the shapes of thestreet and building of the image of FIG. 21 being extracted as indicatedby numerals 54, 55 respectively.

[0159] Thus as described above, with this embodiment, when color valuesare transformed into the HSI space, the saturation values are varied inaccordance with the intensity values by converting the h, s and i valuesfor each pixel to a corresponding set of values that are coordinates ofan inverted-conical shape of color space, so that the instability ofvalues of saturation that is a characteristic feature of the transformfrom RGB to HSI values can be reduced, whereby the occurrence of noisein the obtained results can be substantially suppressed, and reliableedge detection can be achieved.

[0160] A fifth embodiment of an image recognition apparatus will bedescribed. The configuration is identical to that of the secondembodiment (shown in FIG. 15).

[0161] The basic operation sequence of this embodiment is identical tothat of the second embodiment, shown in FIG. 16. Steps 11, 12 and 13 areidentical to those of the first embodiment. With this embodiment, theoperation of step 20 of the flow diagram of FIG. 16 differs from that ofthe second embodiment, as follows. In step 20, the pixel values aretransformed from the RGB color space to coordinates of the cylindricalHSI color space shown in FIG. 20, using equation (9). Equation (13)below is then applied to transform the pixel values to the coordinatesof a color space of the double-conical form shown in FIG. 28.$\begin{matrix}\begin{matrix}{{h^{\prime}\left( {x,y} \right)} = {h\left( {x,y} \right)}} \\{{s^{\prime}\left( {x,y} \right)} = {\left( {1 - \frac{{{i\left( {x,y} \right)} - {{max\_ value}/2}}}{{max\_ value}/2}} \right)\quad {s\left( {x,y} \right)}}} \\{{i^{\prime}\left( {x,y} \right)} = {i\left( {x,y} \right)}}\end{matrix} & (13)\end{matrix}$

[0162] The equation (13) effects a transform of each set of coordinatesof a pixel with respect to the cylindrical HSI space, i.e., h(x, y),s(x, y), i(x, y) to a corresponding set of hue, saturation and intensitycoordinates of the double-conical color space of FIG. 28, which will bedesignated as h′(x, y), s′(x, y), i′(x, y) respectively. This transformdoes not produce any change between h(x, y) and h′(x, y), or betweeni(x, y) and i′(x, y). Furthermore, if the value of i(x, y) is near theintensity value which is located midway between the maximum and minimumvalues of intensity (i.e., ½ of the white level value) there is nodifference between each value of s′(x, y) and s(x, y). However as thevalue of i(x, y) becomes greater or smaller than the intermediate value,the value of s′(x, y) is accordingly reduced in relation to s(x, y).

[0163] The operation of this embodiment for generating respective colorvectors corresponding to the pixels of the color image is shown in moredetail in the flow diagram of FIG. 30. This differs from thecorresponding flow diagram of FIG. 11 for the first embodiment in thatthe step 1002 of the first embodiment, for deriving the array of colorvectors PV which are to be operated on by applying the edge templates inequation (2) as described above to obtain the edge vectors EV1(x,y) toEV2(x,), is divided into four steps, 1002 a, 1002 c, 1002 g and 1002 h.In step 1002 a, the respective sets of r, g, b values for the objectpixel and its eight adjacent surrounding pixels are obtained from thecolor image data storage section 1, and in step 1002 c each of thesesets of r, g, b values of the RGB color space is converted to acorresponding set of h, s, i values of the cylindrical-shape HSI colorspace shown in FIG. 20. In step 1002 g, each of these sets of h, s, ivalues is converted to a corresponding set of h′, s′, i′ values of thedouble-conical H′S′I′ color space shown in FIG. 28. In step 1002 h, eachof these sets is converted to a corresponding set of three linearcoordinates, i.e., of an orthogonal color space, by applying theprocessing of equation (13).

[0164] The remaining steps of this flow diagram, which are omitted fromFIG. 30, are identical to steps 1003 to 1010 of FIG. 11.

[0165] A specific example will be described in the following. In thesame way as for the third embodiment, it will be assumed that thesimplified aerial photograph of FIG. 21 is the color image data that areto be subjected to recognition processing. Firstly, the RGB values ofthe pixels are converted to HSI values of the cylindrical HSI colorspace, and these are then transformed to H′S′I′ values of thedouble-conical color space. The first and third columns of values inFIG. 26 show the relationship between respective HSI values for each ofthe regions, and the corresponding H′S′I′ values resulting from atransform into the coordinates of the double-conical form of H′S′I′color space.

[0166] The image recognition processing results obtained when thisembodiment is applied to edge detection of the color image representedin FIG. 21 are as shown in FIG. 29. As can be seen, not only is thenoise in the low-intensity regions such as the building face 1 andbuilding face 2 of the image of FIG. 21 reduced, but noise is alsogreatly reduced in high-intensity regions such as the building roof andthe street, with the shapes of the street and building being extractedas indicated by numerals 56, 57 respectively.

[0167] Thus with this embodiment, saturation values are reduce inregions of high or low intensity values, i.e., regions in whichinstability of saturation values can be expected to occur as a result ofthe transform from the RGB to the HSI color space. Hence, theinstability of saturation values can be substantially reduced, so thatnoise caused by these saturation values can be suppressed, and accurateedge detection can be achieved.

[0168] A sixth embodiment of an image recognition apparatus will bedescribed. The configuration is identical to that of the secondembodiment shown in FIG. 15, while the basic operation sequence issimilar to that of the second embodiment, shown in the flow diagram ofFIG. 16. Steps 11, 12 and 13 are identical to those of the firstembodiment, shown in the flow diagram of FIG. 5. Step 10 is basicallysimilar to that of the fourth embodiment. 12. The step of performing thetransform from the RGB color space to a different color space (step 20of FIG. 16) is executed as follows with this embodiment. Firstly, thetransform of the pixel values from sets of r, g, b values of the RGBcolor space to h, s, i values of the cylindrical HSI color space of FIG.20 is performed, using equation (9) as described hereinabove for thepreceding embodiment. With the sixth embodiment of the invention, therespective sets of h, s, i values derived for the pixels of the colorimage are then converted to coordinates of a modified H′S I′ color spaceby applying a saturation value modification function, which varies inaccordance with the actual changes in the degree of sensitivity of thesaturation values to small changes in intensity values. This function isgenerated and utilized as follows:

[0169] (1) The first step is to derive, for each of the possible valuesof intensity i, all of the sets of (r, g, b) values which will generatethat value of i when the transform from the RGB to HSI color space isperformed. That is, for each intensity value i(n), where n is in therange from the minimum to maximum (e.g., 255) values, a correspondinggroup of sets of (r, g, b) values are derived.

[0170] (2) For each intensity value, a corresponding set of values of afunction which will be designated as f1(r,g,b) are derived. Theseexpress, for each of the sets of (r, g, b) values, the amount of changewhich would occur in the corresponding value of saturation s, if thevalue of the red component r were to be altered in the range ±1. Eachvalue of f1(r,g,b) is calculated as follows: $\begin{matrix}\begin{matrix}{{{f1}\left( {r,g,b} \right)} = {{{{{s\left( {{r + 1},g,b} \right)} - {s\left( {r,g,b} \right)}}}\quad {if}\quad r} = 0}} \\{{{f1}\left( {r,g,b} \right)} = \frac{\begin{matrix}{{{{s\left( {{r + 1},g,b} \right)} - {s\left( {r,g,b} \right)}}}\quad +} \\{{{{s\left( {r,g,b} \right)} - {s\left( {{r - 1},g,b} \right)}}}\quad}\end{matrix}}{2}} \\{{{if}\quad 0} < r < {max\_ value}} \\{{{f1}\left( {r,g,b} \right)} = {{{{{s\left( {r,g,b} \right)} - {s\left( {{r - 1},g,b} \right)}}}\quad {if}\quad r} = 0}}\end{matrix} & \left( {14a} \right)\end{matrix}$

[0171] (3) Next, for each of the possible values of intensity i, theaverage of the corresponding set of values of f1(r,g,b) is obtained,i.e., a function of i is obtained which will be designated as f2(i).Designating the total number of sets of (r,g,b) values corresponding toa value of intensity i as k(i), this can be expressed as:$\begin{matrix}{{{f2}(i)} = \frac{\sum\limits_{\substack{({{{all}\quad {combinati}\quad {ons}\quad {of}\quad r},g,{b\quad {values}}}\quad \\ \quad {{which}\quad {result}\quad i\quad n\quad {intensity}\quad {value}\quad i})}}{{f2}\quad \left( {r,g,b} \right)}}{k(i)}} & \left( {14b} \right)\end{matrix}$

[0172] where Σf2(r,g,b) signifies, for each value of i, the sum of allof the values obtained as f2(i) for that value of i, i.e., derived fromall of the k sets of (r,g,b) value combinations which will result inthat value of i when a transform from RGB to HSI coordinates isperformed.

[0173] (4) The required saturation value modification function f(i) isthen obtained as follows, designating the minimum value obtained forf2(i) as min f2(i), and the maximum possible value of i as max_value:$\begin{matrix}{{f(i)} = {\frac{\min \quad {{f2}(i)}}{{f2}(i)}\quad {max\_ value}}} & \left( {14c} \right)\end{matrix}$

[0174] The function f(i) is shown in FIG. 31. The higher the value off(i) obtained from equation (14c) above, the greater will be thestability of the s values with respect to changes in the value of thered component r, and the function is derived on the assumption that suchstability also corresponds to stability with respect to changes in theintensity component i. Conversely, the lower the value of f(i), thegreater will be the degree of instability of s of with respect tochanges in the value of r, and hence with respect to changes in thevalue of i.

[0175] That is to say, it is assumed that the values of saturation swill tend to be unstable in regions of the color image where the valuesof the red component r are high, and also in regions where the values ofr are low. Next, using equations (15) below, the respective sets of h,s, i values of the HSI cylindrical color space derived for the pixels ofthe color image are transformed into corresponding sets of coordinatesh′,s′,i′ of the modified cylindrical type of color space shown in FIG.32, by applying the function f(i) derived above. It can be understoodthat the shape of this modified cylindrical color space is formed byrotating the graph of the function f(i) shown in FIG. 31 about itsi-axis. $\begin{matrix}\begin{matrix}{{h^{\prime}\left( {x,y} \right)} = {h\left( {x,y} \right)}} \\{{s^{\prime}\left( {x,y} \right)} = {\frac{f\left( \left( {i\left( {x,y} \right)} \right) \right.}{max\_ value}{s\left( {x,y} \right.}}} \\{{i^{\prime}\left( {x,y} \right)} = {i\left( {x,y} \right)}}\end{matrix} & (15)\end{matrix}$

[0176] The operation of this embodiment for generating respective colorvectors corresponding to the pixels of the color image is shown in thepartial flow diagram of FIG. 34. This differs from the correspondingflow diagram of FIG. 11 for the first embodiment in that the step 1002of the first embodiment, for deriving the array of color vectors PV isreplaced by a series of four steps, 1002 a, 1002 c, 1002 i and 1002 j.In step 1002 a, the respective sets of r, g, b values for the objectpixel and its eight adjacent surrounding pixels are obtained from thecolor image data storage section 1, and in step 1002 c each of thesesets of r, g, b values of the RGB color space is converted to acorresponding set of h, s, i values of the cylindrical-shape HSI colorspace shown in FIG. 20. In step 1002 i, each of these sets of h, s, ivalues is converted to a corresponding set of h′, s′, i′ values of themodified conical H′S′I′ color space shown in FIG. 32, by applyingequation (15). In step 1002 j, each of these sets is converted to acorresponding set of three linear coordinates, i.e., of an orthogonalcolor space, while each of the resultant s′. cos h′ and s′. sin h′values is multiplied by the control parameter “a”, as indicated byequation (12).

[0177] The remaining steps of this flow diagram, which are omitted fromFIG. 34, are identical to steps 1003 to 1010 of FIG. 11.

[0178] A specific example will be described in the following. In thesame way as for the third embodiment, it will be assumed that thesimplified aerial photograph of FIG. 21 constitutes the color image datathat are to be subjected to recognition processing.

[0179] With this embodiment, step 20 of FIG. 16, for conversion to adifferent color space, is executed as follows. The RGB values of thepixels are converted to respective sets of h, s, i values of thecylindrical HSI color space of FIG. 20, and these are then transformedto h′, S′, i′ coordinates of the modified cylindrical color space shownin FIG. 32, by applying the aforementioned function f(i). The contentsof the first and fourth columns of values in the table of FIG. 26 showthe relationship between respective HSI values for each of the regionsof the color image of FIG. 21, and the corresponding H′S′I′ valuesresulting from a transform into the coordinates of the modifiedcylindrical color space.

[0180]FIG. 33 shows the results of image recognition processing obtainedwhen this embodiment is applied to the color image represented in FIG.21. As shown, in addition to reducing noise in regions of low intensity,such as the building face 1 and the building face 2, noise is greatlyreduced in regions of high intensity such as the building roof and theroad. In addition, the shapes of the road and building are veryaccurately obtained, as indicated by numerals 58 and 59 respectively,without any interruptions in the continuity of the edges.

[0181] It can thus be understood that with this embodiment, when thecolor values of the image are transformed from the RGB to respectivesets of h, s, i values that are coordinates of an HSI color space, thesecoordinates are then modified by applying a predetermined function suchthat the intensity values are appropriately reduced in those regions ofthe image where instability of the saturation values would otherwiseoccur. The function which is utilized for performing this modificationof the intensity values is derived on the basis of calculating actualamounts of variation in saturation value that will occur in response tospecific small-scale changes in one of the r, g, or b values, for eachpoint in the RGB color space.

[0182] Hence, compensation of the intensity values is applied in anoptimum manner, i.e. by appropriate amounts, and only to those regionswhere instability of the saturation values would otherwise occur. Thisenables the generation of noise to be effectively suppressed, while atthe same time enabling accurate detection of edges to be achieved, sincethe stability of saturation values is achieved while ensuring that themaximum possible amount of contribution to the magnitude of each colorvector will be made by the corresponding set of h′, s′ and i′ values.That is to say, the maximum possible amount of color information is usedin the edge detection processing, consistent with stability of thesaturation values and resultant elimination of noise from the edgedetection results.

[0183] A seventh embodiment of an image recognition apparatus is shownin FIG. 35. The apparatus is made up of a region data storage section 4having shape data which express only respective regions of an image,i.e. formed of labelled outlines of regions appearing in an image, suchas are generated by the preceding embodiments) with that labelled imagebeing referred to in the following as a region image, an imagerecognition processing section 2 for performing image recognition ofimage data, and a combination-processed shape data storage section 5 forstoring modified shape data which have been formed by the imagerecognition processing section 2 through combining of certain ones ofthe regions expressed in the shape data held in the region data storagesection 4.

[0184] It should be understood that the term “image recognition” asapplied herein to the operation of the image recognition processingsection 2 signifies a form of processing for recognizing certain regionswithin an image which should be combined with other regions of thatimage, and executing such processing.

[0185] As shown in FIG. 35 the image recognition processing section 2 isformed of a small region detection section 26, a combination objectregion determining section 27 and a region combination processingsection 28. The small region detection section 26 performs selection ofcertain regions of the image whose shape data are held in the regiondata storage section 4, based upon criteria described hereinafter. Thecombination object region determining section 27 determines those of theregions selected by the small region detection section 26 which are tobe mutually combined, and the region combination processing section 28performs the actual combination of these regions. The combination objectregion determining section 27 includes a small region determiningsection, which compares the lengths of the respective common borderlines between a selected region and each of the regions which areimmediately adjacent to that selected region, and determines the one ofthese adjacent regions which has the greatest length of common borderline with respect to the selected region.

[0186]FIG. 36 shows an example of a region image whose data are storedin the region data storage section 4. Labels such as “1” and “2” areattached to each of the pixels, as shown in the left side of FIG. 36.All of the pixels located within a specific region have the same label,i.e., there is a region containing only pixels having the label 1, aregion containing only pixels having the label 2, and so on.

[0187] Various techniques are known for separating the contents of animage into various regions. One method of defining a region is to selecta pixel in the image, determine those immediately adjacent pixels whosecolor attributes are sufficiently close to those of the first pixel,within a predetermined range, and to successively expand this processoutwards, to thereby determine all of the pixels which constitute oneregion. Another method is to apply edge detection processing to theimage, and to thereby define each region as a set of pixels which areenclosed within a continuously extending edge.

[0188] With this embodiment, there is no particular limitation on theprocess of generating the region image that is stored in the region datastorage section 4.

[0189] The fundamental feature of the embodiment is that selected smallregions, which constitute noise in the image that is stored in theregion data storage section 4, are combined with adjacent largerregions, or small regions are mutually combined, to thereby eliminatethe small regions and so reduce the level of noise in the region image.Two regions are combined by converting the pixel labels of one of theregions to become identical to the labels of the other region. Theresultant region data, which express the shapes of objects asrespectively different regions, are then stored in thecombination-processed shape data storage section 5.

[0190]FIG. 37 is a flow diagram showing the basic features of theoperation of this embodiment. The contents are as follows. Step 70: adecision is made as to whether there is a set of one or more smallregions within the image which each have an area which is smaller than spixels, where s is a predetermined threshold value. If such a region isfound, then operation proceeds to step 71. If not, i.e., if it is judgedthat all small regions have been eliminated, then operation is ended.Step 71: a region r is arbitrarily selected, as the next small regionthat is to be subjected to region combination, from among the set ofsmall regions which each have an area that is less than s pixels. Step72: for each of the regions r1, r2, . . . rn that are respectivelyimmediately adjacent to the region r, the length of common boundarybetween that adjacent region and the region r is calculated. Step 73:the region ri that is immediately adjacent to the region r and has thelongest value of common boundary line with the region r is selected.Step 74: the regions r and ri are combined to form a new region r′.

[0191] A specific example will be described. It will be assumed that theregion combination processing is to be applied to the region image thatis shown in the upper part of FIG. 38. The image contains regions R, R1,R2 and R3. A vehicle 102 is represented by region R, while a street 100is represented by the region R1. Since the area of the region R is lessthan s pixels, this region is to be deleted.

[0192] There are two regions which are respectively immediately adjacentto the region R, i.e., the regions R1 and R2. The respective lengths ofcommon boundary line between these regions R1, R2 and the region R areobtained, and it is found that the length of common boundary line withrespect to the region R1 is longer than that with respect to R2. Theregion R1 is therefore selected to be combined with the region R. R andR1 are then combined to form a new region, which is designated as R1′,as shown in the lower part of FIG. 38. In that way, the regionrepresenting a vehicle has been removed from the region image whose datawill be stored in the combination-processed shape data storage section5.

[0193] It can be understood that if the pixel values (of the originalcolor image corresponding to the region image) within the region R wereclose to those in the region R2, i.e., if these two regions were closelysimilar in color, and the regions R and R2 were to be combined on thebasis of their closeness of color values, this would result in thestreet attaining an unnatural shape.

[0194] With the embodiment described above, a color image that hasalready been divided into regions is subjected to processing withoutconsideration of the pixel values in the original color image, i.e.,processing that is based only upon the shapes of regions in the image,such as to combine certain regions which have a common boundary line. Asa result, small regions which constitute noise can be removed, withoutlowering the accuracy of extracting shapes of objects which are to berecognized. In particular, in the case of processing image data of anaerial photograph of a city, it is possible to eliminate the shapes ofvehicles on streets, without lowering the accuracy of extracting theshapes of the streets.

[0195] An eighth embodiment of an image recognition apparatus will bedescribed. The configuration is identical to that of the seventhembodiment (shown in FIG. 35).

[0196] The operation sequence of this eighth embodiment is shown in FIG.40. This operation is basically similar to that of the seventhembodiment, shown in the flow chart of FIG. 37, with steps 70, 72, 73,74 being identical to those of the seventh embodiment, however thecontents of step 71 are replaced by those of step 171 in FIG. 40.Specifically, in step 171 of this embodiment, the region r having thesmallest area of all of the regions of the image which have an area ofless than s pixels (as determined in step 70) is selected, and step 72is then applied to that region r.

[0197] A specific example will be described in the following. It will beassumed that the region image shown in the upper part of FIG. 39,representing a building 109 surrounded by a ground area, is to besubjected to combination processing for extracting only the shape of thebuilding roof. There are four regions in the image, R1, R2, R3 and R4with R4 being the ground, R3 being a part of the roof of the building109 which is not covered by rooftop structures, and R1, R2 beingrespective regions corresponding to first and second rooftop structures110, 111 which are formed upon the roof of building 109. The areas ofeach of R1 and R2 is less than s pixels. Since R1 has the smallest areaof all of the regions that are smaller than s pixels, as shown in themiddle portion of FIG. 39, R1 and R3 are combined to obtain the regionR3′. As a result, R2 becomes the region having the smallest area, of theregions R2, R3′ and R4. Hence, R2 and R3′ are combined, to generate aregion R3″. Since the size of each of the remaining regions R3″ and R4is greater than s pixels, the combining processing operation is thenhalted.

[0198] In that way, the rooftop structures on the building areeliminated from the image, so that only the shape of the building itselfwill be extracted.

[0199] It should be noted that if this combining of regions had beenexecuted in the sequence R2, R1, with R2 being combined with R4 and R1being combined with R3, it would be impossible to accurately extract theshape of the building.

[0200] Thus with this embodiment, combining processing is repetitivelyapplied to each of the regions that are below a predetermined size, suchas to combine the region having the smallest area with another region.As a result, small regions which constitute noise can be removed,without lowering the accuracy of extracting shapes for the purpose ofobject recognition. In particular, in the case of applying suchprocessing to image data of an aerial photograph of a city, (i.e., inwhich, as opposed to the usual type of housing, there will frequently becomplex structures formed upon the roofs of buildings) this embodimentwill enable the shapes of the buildings to be accurately extracted.

[0201] A ninth embodiment of an image recognition apparatus will bedescribed. The configuration is identical to that of the seventhembodiment (shown in FIG. 35).

[0202] The operation sequence of this ninth embodiment is shown in theflow diagram of FIG. 42. This is basically similar to that of theseventh embodiment shown in the flow chart of FIG. 37, with steps 70,72, 73, 74 being identical to those of the seventh embodiment. Howeverwith this ninth embodiment, step 71 of FIG. 37 is replaced by twosuccessive steps 271 a, 271 b, executed as follows.

[0203] Step 271 a: for each region having an area that is smaller than spixels, where s is the aforementioned threshold value, the total of theareas of all of the immediately adjacent regions is obtained.

[0204] Step 271 b: the region r, for which the total of the areas of theimmediately adjacent regions is a minimum, is selected to be processedin step 72.

[0205] A specific example will be described in the following. It will beassumed that the region in the upper part of FIG. 41 is to be subjectedto combination processing. There are four regions in the image, R1, R2,R3 and R4, with R4 being the surrounding ground, R1 and R2 are regionscorresponding to first and second structures 112, 113 formed on the roofof building 109, and R3 is the region of that roof which is not coveredby these structures. The area of each of R1 and R2 is less than spixels. The aforementioned sums of areas of immediately adjacent regionsare obtained as follows. The sum of the areas which are immediatelyadjacent to R1 is the total area of R2 and R3, while the sum of suchadjacent areas, in the case of R2, is the total area of R1, R3 and R4.Of these two total areas of adjacent regions, the smaller of the twovalues is obtained for the case of region R1. Thus, as shown in themiddle part of FIG. 41, the regions R3 and R1 are combined to form theregion R3′. In the next repetition of step 71, it is found that there isonly a single region which is smaller than s pixels, and that this isimmediately adjacent to the regions R3′ and R4. Since R3′ is the smallerof these adjacent regions, R3 and R3′ are combined to form a region R3″.Since the size of that region is greater than s pixels, the combiningprocessing operation is then halted.

[0206] In that way, the structures on the building roof having beeneliminated, leaving only the outline of the building roof itself.

[0207] It should be noted that if this combining of regions had beenexecuted in the sequence R2, R1, with R2 being combined with R4 and R1being combined with R3, it would be impossible to accurately extract theshape of the building.

[0208] Thus with this embodiment, combining processing is repetitivelyexecuted such as to combine the region which is below the thresholdvalue of size (s pixels) and for which the total area of the immediatelyadjacent regions is the smallest, with another region. As a result,small regions which constitute noise can be removed, without loweringthe accuracy of extracting shapes for the purpose of object recognition.In particular in the case of applying such processing, whereby combiningprocessing successively occurs from the interior of the outline of abuilding to the periphery of the building, to image data of an aerialphotograph of a city in which there will be many complex rooftopconfigurations, this embodiment will enable the shapes of the buildingsto be accurately extracted.

[0209] In the description of the preceding embodiments it has beenassumed that the small region detection section 26 shown in FIG. 5determines the regions which are to be classified as part of the set ofsmall regions (i.e., that are to be subjected to region combinationprocessing) based upon whether or not the total area of a region isabove a predetermined threshold value (s pixels). However it should benoted that the invention is not limited to this method, and other typesof criteria for selecting these small regions could be envisaged,depending upon the requirements of a particular application. Forexample, it might be predetermined that regions which are narrower thana predetermined limit are to be combined with other regions,irrespective of total area. It should thus be understood that variousmodifications to the embodiments described above could be envisaged,which fall within the scope claimed for the present invention.

[0210] A tenth embodiment of an image recognition apparatus according tothe present invention will be described. As shown in FIG. 43, this isformed of a color image data storage section 1 which stores color imagedata, an image recognition processing section 2 for performing imagerecognition processing of the color image data, and acombination-processed shape data storage section 5 for storing shapedata expressing a region image, extracted by the image recognitionprocessing section 2.

[0211] The image recognition processing section 2 of this embodiment ismade up of a color space coordinates conversion section 25, color vectordata generating section 21, edge template application section 22, edgestrength and direction determining section 23, an edge pixel determiningsection 24 for extracting shape data expressing an edge image asdescribed hereinabove referring to FIG. 16, a small region detectionsection 26, a combination object region determining section 27, and aregion combination processing section 28 for performing region combiningprocessing as described hereinabove referring to FIG. 35, and an edgedata-region data conversion section 29.

[0212] The color space coordinates conversion section 25 converts theRGB data that are stored in the color image data storage section 1 tocoordinates of an appropriate color space (i.e., whereby intensity andchrominance information are expressed respectively separately). Thecolor vector data generating section 21 generates respective colorvectors, each expressed by a plurality of scalar value, corresponding tothe pixels of the original color image, from the transformed image data.The edge template application section 22 applies edge templates to thepixel vector data, to generate edge vector data. The edge strength anddirection determining section 23 determines the edge strength and theedge direction information, based on the magnitudes of the edge vectormoduli, as described hereinabove for the first embodiment, with the edgepixel determining section 24 determining those pixels which are locatedon edges within the color image, based on the edge strength anddirection information, to thereby obtain shaped data expressing an edgeimage. The edge data-region data conversion section 29 converts the edgeimage data into shape data expressing a region image. The small regiondetection section 26 selects a set of small regions which are each to besubjected to region combination processing, and the combination objectregion determining section 27 determines the next one of that set ofsmall regions that is to be subjected to the region combinationprocessing. The combination object region determining section 27operates on that small region, to determine the respective lengths ofthe common border lines between that small region and each of itsimmediately adjacent regions, and combines the small region with theadjacent region having the greatest length of common border line withthe small region.

[0213]FIG. 44 is a flow diagram of the operating sequence of theapparatus of the embodiment of FIG. 10.

[0214] The processing of the sequence of steps 20, 10, 11, 12, and 13 isidentical to that shown in FIG. 16 of the second embodiment, describedhereinabove, so that detailed description will be omitted. Similarly,the processing executed in the sequence of steps 70, 72, 73, 74 isidentical to shown in FIG. 37 for the seventh embodiment. In step 100,the data expressing the edge image are converted to data expressing aregion image. This is done by dividing the edge image into regions, eachformed of a continuously extending set of pixels that are surrounded byedge pixels, and applying a common label to each of the pixels of such aregion as described hereinabove referring to FIG. 36, i.e., applyingrespectively different labels to identify the various regions.

[0215] A specific example will be described, assuming that thesimplified aerial photograph which is represented in the upper part ofFIG. 45 is the color image whose data that are to be subjected torecognition processing by this embodiment. This image contains a road122, two vehicles 121 and a building 120. When edge detection is appliedto this image, using respective pluralities of scalar values of thepixels of the color image data, the results are as shown in the middlepart of FIG. 45. As shown, edge data are detected for the road, thevehicles and the building, respectively, so that the shapes 123 of thevehicles appear in the street. The data of that edge image are thenconverted to data of a region image as described above, and regioncombining is applied based upon the shapes of the regions, withoutconsideration of the values of pixels within the regions. The resultobtained is as shown in the lower part of FIG. 45. As shown, thevehicles have been eliminated, leaving the shape 124 of the roadaccurately represented.

[0216] The upper part of FIG. 46 shows an edge image that has beenobtained by applying edge detection by an embodiment of the presentinvention to a color image which is an actual aerial photographcontaining various roads and buildings and many vehicles. Numeral 130indicates various small regions appearing in the edge image whichcorrespond to the outlines of respective vehicles, while the largerrectangular regions designated by numeral 131 correspond to buildings.In the original photograph there is almost no difference in intensitybetween the building roofs and the surrounding ground surface. Hence, ifprior art methods of image recognition were to be applied in thisinstance, it would be difficult to detect the shapes of the edges of thebuildings. However by applying the present invention, the building edgesare accurately detected.

[0217] The edge image is then converted to a region image, and regioncombination is applied to that region image as described above, i.e.,with the combination processing being based upon the shapes of theregions, without consideration of the values of pixels within theregions, and with the aforementioned threshold value s being set to anappropriate value for substantially eliminating the small regions 130which correspond to vehicles.

[0218] The result obtained is as shown in the lower part of FIG. 46. Asshown, the shapes of many vehicles have been eliminated, therebyenabling the buildings to be more easily recognized, without reducingthe accuracy of extracting the shapes of the buildings.

[0219] As can be understood from the above description of embodiments,according to one basic aspect, the present invention provides an imagerecognition method and image recognition apparatus whereby the edges ofregions expressing objects appearing in a color image can be accuratelyand reliably detected. This is based upon expressing the colorattributes of each pixel of the image as a plurality of scalar valuesexpressing a color vector, and the use of edge vectors corresponding torespective ones of a plurality of predetermined edge directions (i.e.,specific orientation angles within an image). The pixels of the colorimage are selective processed to derive a corresponding set of edgevectors, with each edge vector being a vector quantity which isindicative of an amount of variation in color between pixels which arelocated on opposite sides of a line extending through the selected pixeland extending in the corresponding edge direction. Each edge vector isderived in a simple manner by performing an array multiplicationoperation between an edge template and an array of color vectorscentered on the selected pixel, and obtaining the vector sum of theresult. With the described embodiments, this operation is equivalent toselecting first and second sets of pixels that are located onrespectively opposing sides of the selected pixel, with respect to aspecific edge direction, obtaining respective weighted vector sums ofthe color vectors of these two sets, and obtaining the vector differencebetween these sums. The edge direction corresponding to the edge vectorhaving the largest modulus of the resultant set of edge vectors obtainedfor the selected pixel (that largest value being referred to as the edgestrength) is thereby obtained as the most probable edge direction onwhich that pixel is located, and it thereby becomes possible to reliablydetect those pixels which actually are located on edges, based oncomparisons of respective values of edge strength of adjacent pixels,and also to obtain the direction of such an edge.

[0220] According to a second basic aspect of the invention, a regionimage which expresses an image as a plurality of respectively identifiedregions can be processed to eliminate specific small regions which arenot intended to be identified, and which therefore constitute noise withrespect to an image recognition function. This is achieved by firstdetecting the set of small regions which are each to be eliminated bybeing combined with an adjacent region, then determining the next one ofthat set which is to be subjected to the combination processing, withthat determination being based upon specific criteria which are designedto prevent the combination of the small regions having the effect ofdistorting the shapes of larger regions which are to be recognized. Thesmall region thus determined is then combined with an adjacent region,with that adjacent region also being selected such as to reduce thepossibility of distortion of regions which are intended to berecognized. In that way, the disadvantages of prior art methods ofreducing such small regions, such as by various forms of filterprocessing, can thereby be effectively overcome.

What is claimed is:
 1. An image recognition method of processing imagedata of a color image which is represented as respective sets of colorattribute data of an array of pixels, to successively operate on each ofsaid pixels as an object pixel for thereby determining whether saidobject pixel is located on an edge within said color image, and therebyderive shape data expressing an edge image corresponding to said colorimage, the method comprising steps of: expressing said sets of colorattribute data of each of said pixels as respective color vectors, witheach said color vector defined by a plurality of scalar values which arecoordinates of an orthogonal color space; for each of a plurality ofpredetermined edge directions, generating a corresponding edge templateas an array of respectively predetermined numeric values; extracting anarray of color vectors a respective color vectors of an array of saidpixels, said array of pixels being centered on said object pixel;successively applying each of said edge templates to said array of colorvectors in a predetermined array processing operation, to derive edgevectors respectively corresponding to said edge directions; comparingthe respective moduli of said derived edge vectors to obtain a value ofedge strength for said object pixel, as a maximum value of modulus ofsaid edge vectors, and obtaining a possible edge direction for saidobject pixel as a direction corresponding to an edge vector having saidmaximum value of modulus; and judging whether said object pixel islocated on an actual edge which is oriented in said possible edgedirection, based upon comparing said edge strength of said object pixelwith respective values of edge strength derived for pixels disposedadjacent to said object pixel.
 2. The image recognition method accordingto claim 1, wherein said step of judging whether said object pixel islocated on an actual edge which is oriented in said possible edgedirection comprises comparing said edge strength of said object pixelwith a predetermined threshold value and with respective values of edgestrength of first and second adjacent pixels, said first and secondadjacent pixels being located immediately adjacent to said object pixeland on opposing sides of said object pixel with respect to said possibleedge direction, and judging that said object pixel is located on anactual edge which is oriented in said possible edge direction when it isfound that said edge strength of said object pixel exceeds saidthreshold value and also exceeds said respective values of edge strengthof said first and second adjacent pixels.
 3. The image recognitionmethod according to claim 1, wherein said numeric values constitutingeach of said edge templates include positive and negative values whichare respectively disposed symmetrically opposite in relation to saidcorresponding edge direction within said edge template, and wherein saidstep of applying an edge template comprises performing an arraymultiplication operation between said edge template and said array ofcolor vectors, and obtaining the vector sum of a result of said arraymultiplication operation as an edge vector.
 4. The image recognitionmethod according to claim 1, wherein said step of comparing the moduliof said derived edge vectors to obtain said value of edge strength ofsaid object pixel comprises: based on results of said comparison,selectively determining that said moduli have a first relationshipwhereby there is only a single maximum one of said modului, a secondrelationship whereby all of said moduli have an identical value, or athird relationship whereby a plurality of said moduli are greater thanremaining one(s) of said moduli; when said first relationship isdetermined, registering said maximum modulus as said value of edgestrength of said object pixel, and registering information specifying adirection corresponding to the edge vector having said maximum modulusas the possible edge direction of said object pixel; when said secondrelationship is determined, registering said identical value of modulusas said value of edge strength of said object pixel; and when said thirdrelationship is determined, arbitrarily selecting an edge vector havingsaid greater value of modulus, registering said modulus value as saidvalue of edge strength of said object pixel, and registering informationfor specifying a direction which corresponds to said selected edgevector as the possible edge direction of said object pixel.
 5. The imagerecognition method according to claim 1, wherein said step of comparingthe moduli of said derived edge vectors to obtain said value of edgestrength of said object pixel comprises: based on results of saidcomparison, selectively determining that said moduli have a firstrelationship whereby there is only a single maximum one of said moduli,a second relationship whereby all of said moduli have an identicalvalue, or a third relationship whereby a plurality of said moduli aregreater than remaining one(s) of said moduli; when said firstrelationship is determined, registering said maximum modulus as saidvalue or edge strength of said object pixel, and registering informationspecifying a direction corresponding to the edge vector having saidmaximum modulus, as a single candidate edge direction of said objectpixel; when said second relationship is determined, registering saididentical value of modulus as said value of edge strength of said objectpixel; and when said third relationship is determined, registering saidgreater value of modulus as said value of edge strength of said objectpixel, and registering information specifying each of respectivedirections corresponding to each of said plurality of edge vectorshaving said greater value of modulus, as respective candidate edgedirections of said object pixel; and wherein said step of judgingwhether said object pixel is located on an actual edge is performed bysuccessively utilizing each of said candidate edge directions, until anactual edge is detected or all of said candidate edge directions havebeen utilized.
 6. The image recognition method according to claim 1,wherein said step of expressing said sets of color attribute data asrespective color vectors comprises performing a transform processingoperation on each of said sets or color attribute data to derive acorresponding plurality of scalar values which constitute a set ofcoordinates of a predetermined color space.
 7. The image recognitionmethod according to claim 6, wherein said predetermined color space isan HSI (hue, saturation, intensity) color space.
 8. The imagerecognition method according to claim 7, wherein said coordinates ofsaid HSI color space are obtained in the form of polar coordinates, andfurther comprising a step of converting each said set of polarcoordinates to a corresponding plurality of scalar values which arelinear coordinates of an orthogonal color space.
 9. The imagerecognition method according to claim 8, wherein said set of linearcoordinates obtained corresponding to each of said pixels is derivedsuch that an intensity value for said pixel is expressed by a specificone of said set of coordinates while hue and saturation values for saidpixel are expressed by other ones of said set of coordinates, andfurther comprising a step of multiplying at least one of saidcoordinates of said set by an arbitrarily determined parameter valuesuch as to alter a relationship between respective magnitudes of saidintensity value and said hue and saturation values.
 10. The imagerecognition method according to claim 7, further comprising a step ofconverting each of said sets of coordinates of said pixels for said HSIcolor space to a corresponding set of coordinates of a modified HSIcolor space, such that saturation values expressed in said modified HSIcolor space are modified in accordance with corresponding intensityvalues.
 11. The image recognition method according to claim 10, whereinsaid saturation values in the modified HSI color space are decreased inaccordance with decreases in corresponding intensity values, in relationto saturation values in said HSI color space.
 12. The image recognitionmethod according to claim 10, wherein said saturation values in themodified HSI color space are decreased in relation to saturation valuesin said HSI color space, in accordance with increases in correspondingintensity values from a predetermined median intensity value, and aremoreover decreased in relation to saturation values in said HSI colorspace in accordance with decreases in corresponding intensity valuesfrom said predetermined median intensity value.
 13. The imagerecognition method according to claim 10, wherein said step ofconverting each of said sets of coordinates of said pixels for said HSIcolor space to a corresponding set of coordinates of the modified HSIcolor space comprises applying a predetermined modification function toeach of respective saturation values of said HSI color space to obtainmodified saturation values.
 14. The image recognition method accordingto claim 13, wherein said modification function is derived beforehandbased upon a relationship between the intensity values and correspondingsaturation values which are obtained by a transform into an HSI spacehaving a specific size, with each of respective hue, saturation andintensity values expressed as a specific number of data bits.
 15. Amethod of deriving for a selected pixel of a color image which is formedof an array of pixels, for each of a plurality of predetermined edgedirections, an edge strength value which corresponds to a specific oneof a plurality of predetermined edge directions and is indicative of adegree of probability that said selected pixel is located on an edgebetween regions of respectively different color within said image, withsaid edge being oriented in said specific edge direction, the methodcomprising a set of steps performed for each of said edge directions of:expressing the color attributes of each of said pixels of said colorimage as a plurality of scalar values representing a color vector withinan orthogonal color space; obtaining a first weighted vector sum of afirst set of pixels which are located adjacent to said selected pixel onone side thereof with respect to said specific edge direction and asecond weighted vector sum of a second set of pixels which are locatedadjacent to said selected pixel on an opposite side from said first setwith respect to said specific edge direction, and deriving the vectordifference between said first and second weighted vector sums; andobtaining the modulus of said vector difference, and a step of judgingthe respective moduli thereby obtained respectively corresponding tosaid predetermined edge directions, to obtain said edge strength valueas the largest one of said moduli.
 16. An image recognition method foroperating on shape data expressing an original region image to obtainshape data expressing a region image in which specific small regionshave been eliminated, comprising repetitive execution of a series ofsteps of: selectively determining respective regions of said originalregion image as constituting a set of small regions which are each to besubjected to a region combining operation; selecting one of said set ofsmall regions as a next small region which is to be subjected to saidregion combining operation; for each of respective regions which aredisposed immediately adjacent to said next small region, calculating alength of common boundary line with respect to said next small region,and determining one of said immediately adjacent regions which has amaximum value of said length of boundary line; and combining said nextsmall region with said adjacent region having the maximum length ofcommon boundary line.
 17. The image recognition method according toclaim 16, wherein said step of determining said set of small regionswhich are each to be subjected to a region combining operation isperformed based upon judgement of respective size values of each of saidregions of said color image.
 18. The image recognition method accordingto claim 16, wherein said step of determining said set of small regionswhich are each to be subjected to a region combining operation isperformed by selecting each of said regions of said color image havingan area which is less than a predetermined threshold value.
 19. Theimage recognition method according to claim 16, wherein said step ofselecting one of said set of small regions as a next small region to besubjected to region combination is performed by selecting an arbitraryone of said set of small regions.
 20. The image recognition methodaccording to claim 16, wherein said step of selecting one of said set ofsmall region as a next small region to be subjected to regioncombination is performed by selecting the smallest one of said set ofsmall regions.
 21. The image recognition method according to claim 16,wherein said step of selecting one of said set of small regions as anext small region to be subjected to region combination is based uponthe respective sizes of said set of small regions.
 22. The imagerecognition method according to claim 16, wherein said step of selectingone of said set of small regions as a next small region to be subjectedto region combination is based upon the respective total sizes of setsof regions which are located immediately adjacent to respective ones ofsaid set of small regions.
 23. The image recording method according toclaim 1, further comprising a step of converting said shape dataexpressing said edge image to shape data expressing a correspondingregion image, and repetitive execution of a series of steps of:determining all regions of said original region image which each have asize that is below a predetermined threshold value, as constituting aset of small regions which are each to be subjected to regioncombination; selecting one of said set of small regions as a next smallregion which is to be subjected to said region combination; for each ofrespective regions which are disposed immediately adjacent to said nextsmall region, calculating a length of common boundary line with respectto said next small region, and determining one of said immediatelyadjacent regions which has a maximum value of said length of boundaryline; and combining said next small region with said adjacent regionhaving the maximum length of common boundary line.
 24. The imagerecording method according to claim 6, further comprising a step ofconverting said shape data expressing said edge image to shape dataexpressing a corresponding region image, and repetitive execution of aseries of steps of: determining all regions of said original regionimage which each have a size that is below a predetermined thresholdvalue, as constituting a set of small regions which are each to besubjected to region combination; selecting one of said set of smallregions as a next small region which is to be subjected to said regioncombination; for each of respective regions which are disposedimmediately adjacent to said next small region, calculating a length ofcommon boundary line with respect to said next small region, anddetermining one of said immediately adjacent regions which has a maximumvalue of said length of boundary line; and combining said next smallregion with said adjacent region having the maximum length of commonboundary line.
 25. An image recognition apparatus for processing imagedata of a color image which is represented as respective sets of colorattribute data of an array of pixels, to successively operate on each ofsaid pixels as an object pixel for thereby determining whether saidobject pixel is located on an edge within said color image, and therebyderive shape data expressing an edge image corresponding to said colorimage, the apparatus comprising: color vector generating means forexpressing said sets of color attribute data of each of said pixels asrespective color vectors, with each said color vector in the form of anarray of a plurality of scalar values which are coordinates of anorthogonal color space; edge template application means for generating aplurality of edge templates each formed of an array of respectivelypredetermined numeric values, with said edge templates corresponding torespective ones of a plurality of predetermined edge directions, forextracting an array of color vectors as respective color vectors of anarray of said pixels, with said array of pixels centered on said objectpixel, and successively applying each of said edge templates to saidarray of color vectors in a predetermined array processing operation, toderive edge vectors respectively corresponding to said edge directions;edge pixel determining means for comparing the respective moduli of saidderived edge vectors to obtain a value of edge strength for said objectpixel, as a maximum value of modulus of said edge vectors, for obtaininga possible edge direction for said object pixel as a directioncorresponding to an edge vector having said maximum value of modulus,and for judging whether said object pixel is located on an actual edgewhich is oriented in said possible edge direction, based upon comparingsaid edge strength of said object pixel with respective values of edgestrength derived for pixels disposed adjacent to said object pixel. 26.The image recognition apparatus according to claim 25, wherein saidoperation of judging whether said object pixel is located on an actualedge which is oriented in said possible edge direction comprisescomparing said edge strength of said object pixel with a predeterminedthreshold value and with respective values of edge strength of first andsecond adjacent pixels, said first and second adjacent pixels beinglocated immediately adjacent to said object pixel and on opposing sidesof said object pixel with respect to said possible edge direction, andjudging that said object pixel is located on an actual edge which isoriented in said possible edge direction when it is found that said edgestrength of said object pixel exceeds said threshold value and alsoexceeds said respective values of edge strength of said first and secondadjacent pixels.
 27. The image recognition apparatus according to claim25, wherein said numeric values constituting each of said edge templatesinclude positive and negative values which are respectively disposedsymmetrically opposite in relation to said corresponding edge directionwithin said edge template, and wherein said operation of applying anedge template is executed by performing an array multiplicationoperation between said edge template and said array of color vectors,and obtaining the vector sum of a result of said array multiplicationoperation as an edge vector.
 28. The image recognition apparatusaccording to claim 25, wherein said operation of comparing the moduli ofsaid derived edge vectors to obtain said value of edge strength of saidobject pixel comprises: based on results of said comparison, selectivelydetermining that said moduli have a first relationship whereby there isonly a single maximum one of said moduli, a second relationship wherebyall of said moduli have an identical value, or a third relationshipwhereby a plurality of said moduli are greater than remaining one(s) ofsaid moduli; when said first relationship is determined, registeringsaid maximum modulus as said value of edge strength of said objectpixel, and registering information specifying a direction correspondingto the edge vector having said maximum modulus as the possible edgedirection of said object pixel; when said second relationship isdetermined, registering said identical value of modulus as said value ofedge strength of said object pixel; and when said third relationship isdetermined, arbitrarily selecting an edge vector having said greatervalue of modulus, registering said modulus value as said value of edgestrength of said object pixel, and registering information whichspecifies that a direction corresponding to said selected edge vector isa possible edge direction of said object pixel.
 29. The imagerecognition apparatus according to claim 25, wherein said operation ofcomparing the moduli of said derived edge vectors to obtain said valueof edge strength of said object pixel comprises: based on results ofsaid comparison, selectively determining that said moduli have a firstrelationship whereby there is only a single maximum one of said moduli,a second relationship whereby all of said moduli have an identicalvalue, or a third relationship whereby a plurality of said moduli aregreater than remaining one(s) of said moduli; when said firstrelationship is determined, registering said maximum modulus as saidvalue of edge strength of said object pixel, and registering informationspecifying a direction corresponding to the edge vector having saidmaximum modulus, as a single candidate edge direction of said objectpixel; when said second relationship is determined, registering saididentical value of modulus as said value of edge strength of said objectpixel; and when said third relationship is determined, registering saidgreater value of modulus as said value of edge strength of said objectpixel, and registering information specifying each of respectivedirections corresponding to each of said plurality of edge vectorshaving said greater value of modulus, as respective candidate edgedirections of said object pixel; and wherein said operation of judgingwhether said object pixel is located on an actual edge is performed bysuccessively utilizing each of said candidate edge directions, until anactual edge is detected or all of said candidate edge directions havebeen utilized.
 30. The image recognition apparatus according to claim25, wherein said operation of expressing said sets of color attributedata as respective color vectors is executed by performing a transformprocessing operation on each of said sets of color attribute data toderive a corresponding plurality of scalar values which constitute a setof coordinates of a predetermined color space.
 31. The image recognitionapparatus according to claim 30, wherein said predetermined color spaceis an HSI (hue, saturation, intensity) color space.
 32. The imagerecognition apparatus according to claim 31, wherein said coordinates ofsaid HSI color space are obtained in the form of polar coordinates, andwherein said color vector generating means further comprises means forconverting each said set of polar coordinates to a correspondingplurality of scalar values which are linear coordinates of an orthogonalcolor space.
 33. The image recognition apparatus according to claim 32,wherein said set of linear coordinates obtained corresponding to each ofsaid pixels is derived such that an intensity value for said pixel isexpressed by a specific one of said set of coordinates while hue andsaturation values for said pixel are expressed by other ones of said setof coordinates, and wherein said color vector generating means furthercomprises means for multiplying at least one of said coordinates of saidset by an arbitrarily determined parameter value to thereby alter arelationship between respective magnitudes of said intensity value andsaid hue and saturation values.
 34. The image recognition apparatusaccording to claim 31, wherein said color vector generating meansfurther comprises means for converting each of said sets of coordinatesof said pixels for said HSI color space to a corresponding set ofcoordinates of a modified HSI color space, such that saturation valuesexpressed in said modified HSI color space are altered in accordancewith corresponding intensity values.
 35. The image recognition apparatusaccording to claim 34, wherein said saturation values in the modifiedHSI color space are decreased in accordance with decreases incorresponding intensity values, in relation to saturation values in saidHSI color space.
 36. The image recognition apparatus according to claim34, wherein said saturation values in the modified HSI color space aredecreased in relation to saturation values in said HSI color space, inaccordance with increases in corresponding intensity values from apredetermined median value, and are moreover decreased in relation tosaturation values in said HSI color space, in accordance with decreasesin corresponding intensity values from said predetermined median value.37. The image recognition apparatus according to claim 34, wherein saidoperation of converting each of said sets of coordinates of said pixelsfor said HSI color space to a corresponding set of coordinates of themodified HSI color space is executed by applying a predeterminedmodification function to each of respective saturation values of saidHSI color space to obtain modified saturation values.
 38. The imagerecognition apparatus according to claim 37, wherein said modificationfunction is derived beforehand based upon a relationship between theintensity values and corresponding saturation values which are obtainedby a transform into an HSI space having a specific size, with each ofrespective hue, saturation and intensity values expressed as a specificnumber of data bits.
 39. An image recognition apparatus for operating onshape data expressing an original region image to obtain shape dataexpressing a region image in which specific small regions have beeneliminated, comprising: small region detection means for selectivelydetermining respective regions of said original region image asconstituting a set of small regions which are each to be subjected toregion combination; region combination determining means for selectingone of said set of small regions as a next small region which is to besubjected to said region combination; and, region combining means forcalculating respective values of common boundary line between said nextsmall region and each of the regions which are located immediatelyadjacent to said next small region, for determining one of saidimmediately adjacent regions which has a maximum value of said length ofboundary line, and for combining said next small region with saidadjacent region having the maximum length of common boundary line. 40.The image recognition apparatus according to claim 39, wherein saidsmall region detection means comprises means for determining said set ofsmall regions which are each to be subjected to a region combiningoperation based upon judgement of respective size values of each of saidregions of said color image.
 41. The image recognition apparatusaccording to claim 39, wherein said region combination determining meanscomprises means for determining one of said set of small regions as saidnext small region to be subjected to region combination, by selecting anarbitrary one of said set of small regions.
 42. The image recognitionapparatus according to claim 39, wherein said region combinationdetermining means determines one of said set of small regions, as saidnext small region to be subjected to region combination, by selectingthe smallest one of said set of small regions.
 43. The image recognitionapparatus according to claim 39, wherein said region combinationdetermining means determines one of said set of small regions, as saidnext small region to be subjected to region combination, based upon therespective sizes of said set of small regions.
 44. The image recognitionapparatus according to claim 39, wherein said region combinationdetermining means determines one of said set of small regions, as saidnext small region to be subjected to region combination, based upon therespective total sizes of sets of regions which are located immediatelyadjacent to respective ones of said set of small regions.
 45. The imagerecording apparatus according to claim 39, wherein said small regiondetection means comprises means for selecting respective ones of saidregions of the original region image that are smaller than apredetermined threshold value as said small regions which are to besubjected to region.combination.
 46. The image recording apparatusaccording to claim 25, further comprising: means for converting saidshape data expressing said edge image to shape data expressing a regionimage in which respective regions are separately identified; smallregion detection means for selectively determining respective regions ofsaid region image as constituting a set of small regions which are eachto be subjected to region combination; region combination determiningmeans for selecting one of said set of small regions as a next smallregion which is to be subjected to said region combination; and, regioncombining means for calculating respective values of common boundaryline between said next small region and each of the regions which arelocated immediately adjacent to said next small region, for determiningone of said immediately adjacent regions which has a maximum value ofsaid length of boundary line, and for combining said next small regionwith said immediately adjacent region having the maximum length ofcommon boundary line.
 47. The image recording apparatus according toclaim 30, further comprising: means for converting said shape dataexpressing said edge image to shape data expressing a region image inwhich respective regions are separately identified; small regiondetection means for selectively determining respective regions of saidregion image as constituting a set of small regions which are each to besubjected to region combination; region combination determining meansfor selecting one of said set of small regions as a next small regionwhich is to be subjected to said region combination; and, regioncombining means for calculating respective values of common boundaryline between said next small region and each of the regions which arelocated immediately adjacent to said next small region, for determiningone of said immediately adjacent regions which has a maximum value ofsaid length of boundary line, and for combining said next small regionwith said immediately adjacent region having the maximum length ofcommon boundary line.